Sep 14 07:47:55.118218 ip-10-0-128-30 systemd[1]: Starting Kubernetes Kubelet...
Sep 14 07:47:55.546693 ip-10-0-128-30 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:55.546693 ip-10-0-128-30 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 14 07:47:55.546693 ip-10-0-128-30 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:55.546693 ip-10-0-128-30 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 14 07:47:55.546693 ip-10-0-128-30 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 14 07:47:55.549645 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.549556    2577 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 14 07:47:55.555443 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555427    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.555443 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555443    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.555443 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555447    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555450    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555453    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555456    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555460    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555463    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555466    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555469    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555481    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555484    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555487    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555489    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555491    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555494    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555497    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555499    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555502    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555504    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555507    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555509    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.555539 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555512    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555514    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555517    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555519    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555521    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555524    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555526    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555529    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555531    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555533    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555536    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555539    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555542    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555545    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555548    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555551    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555554    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555557    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555559    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555562    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.556080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555564    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555567    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555569    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555572    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555575    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555577    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555580    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555582    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555585    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555587    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555590    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555592    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555595    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555597    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555600    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555603    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555605    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555608    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555610    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555612    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.556560 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555616    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555618    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555621    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555623    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555626    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555628    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555631    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555633    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555636    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555638    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555641    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555643    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555646    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555649    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555652    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555654    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555657    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555659    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555664    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555668    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.557051 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555670    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555673    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555677    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555682    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555685    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555688    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555691    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555695    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555698    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555701    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555704    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555707    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555710    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555713    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555716    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555718    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.555721    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556413    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556419    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.557551 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556422    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556425    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556428    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556430    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556433    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556437    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556441    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556445    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556448    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556451    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556453    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556456    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556458    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556461    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556464    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556467    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556469    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556472    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556475    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556479    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.558130 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556481    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556484    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556486    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556489    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556492    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556494    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556497    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556500    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556503    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556506    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556509    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556511    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556514    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556516    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556519    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556521    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556524    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556526    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556529    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.558946 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556531    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556535    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556537    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556540    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556543    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556546    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556548    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556551    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556553    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556556    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556559    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556562    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556564    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556567    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556570    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556573    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556575    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556578    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556580    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556585    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.559442 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556589    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556593    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556596    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556599    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556602    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556605    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556607    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556610    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556613    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556616    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556619    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556622    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556625    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556627    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556630    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556633    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556636    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556639    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556641    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556644    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.559950 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556646    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556649    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556651    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556654    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556657    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556659    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556662    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556665    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556667    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556671    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556674    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556677    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556680    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556684    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556687    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556690    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556693    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.556695    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557073    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.560441 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557078    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557081    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557083    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557087    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557090    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557092    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557095    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557098    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557101    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557104    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557107    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557109    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557112    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557115    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557118    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557120    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557123    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557125    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557128    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.560901 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557131    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557134    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557138    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557141    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557143    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557148    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557152    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557155    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557158    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557161    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557164    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557167    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557170    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557173    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557176    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557179    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557181    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557184    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557187    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.561390 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557189    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557191    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557196    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557200    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557203    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557206    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557209    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557211    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557214    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557217    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557219    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557222    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557224    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557226    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557229    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557231    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557233    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557236    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557239    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557241    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.561849 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557244    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557246    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557250    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557252    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557255    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557260    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557263    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557265    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557267    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557270    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557272    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557275    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557277    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557280    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557282    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557285    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557287    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557289    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557292    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557294    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.562346 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557297    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557299    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557302    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557304    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557307    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557310    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557312    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557315    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557317    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557320    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557323    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557326    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557328    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557331    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557333    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557336    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557339    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557343    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557345    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.557348    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557416    2577 flags.go:64] FLAG: --address="0.0.0.0"
Sep 14 07:47:55.562904 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557450    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557456    2577 flags.go:64] FLAG: --anonymous-auth="true"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557461    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557465    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557468    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557473    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557477    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557480    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557483    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557487    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557490    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557493    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557496    2577 flags.go:64] FLAG: --cgroup-root=""
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557499    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557502    2577 flags.go:64] FLAG: --client-ca-file=""
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557505    2577 flags.go:64] FLAG: --cloud-provider="external"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557508    2577 flags.go:64] FLAG: --cluster-dns="[]"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557513    2577 flags.go:64] FLAG: --cluster-domain=""
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557516    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557519    2577 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557523    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557528    2577 flags.go:64] FLAG: --container-log-max-files="5"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557532    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 14 07:47:55.563442 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557534    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557538    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557541    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557544    2577 flags.go:64] FLAG: --contention-profiling="false"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557548    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557552    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557555    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557559    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557563    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557566    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557570    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557573    2577 flags.go:64] FLAG: --enable-load-reader="false"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557576    2577 flags.go:64] FLAG: --enable-server="true"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557579    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557583    2577 flags.go:64] FLAG: --event-burst="100"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557586    2577 flags.go:64] FLAG: --event-qps="50"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557589    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557591    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557595    2577 flags.go:64] FLAG: --eviction-hard=""
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557598    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557601    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557604    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557607    2577 flags.go:64] FLAG: --eviction-soft=""
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557610    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557613    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 14 07:47:55.564051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557616    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557619    2577 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557622    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557625    2577 flags.go:64] FLAG: --fail-swap-on="true"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557628    2577 flags.go:64] FLAG: --feature-gates=""
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557633    2577 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557636    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557639    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557642    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557646    2577 flags.go:64] FLAG: --healthz-port="10248"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557648    2577 flags.go:64] FLAG: --help="false"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557651    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557656    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557659    2577 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557662    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557665    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557669    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557672    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557675    2577 flags.go:64] FLAG: --image-service-endpoint=""
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557678    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557680    2577 flags.go:64] FLAG: --kube-api-burst="100"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557683    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557687    2577 flags.go:64] FLAG: --kube-api-qps="50"
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557689    2577 flags.go:64] FLAG: --kube-reserved=""
Sep 14 07:47:55.564677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557692    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557695    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557698    2577 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557701    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557704    2577 flags.go:64] FLAG: --lock-file=""
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557706    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557710    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557713    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557718    2577 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557721    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557723    2577 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557726    2577 flags.go:64] FLAG: --logging-format="text"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557729    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557734    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557737    2577 flags.go:64] FLAG: --manifest-url=""
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557739    2577 flags.go:64] FLAG: --manifest-url-header=""
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557744    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557746    2577 flags.go:64] FLAG: --max-open-files="1000000"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557751    2577 flags.go:64] FLAG: --max-pods="110"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557753    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557757    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557760    2577 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557763    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557766    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557769    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 14 07:47:55.565295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557772    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557778    2577 flags.go:64] FLAG: --node-status-max-images="50"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557782    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557784    2577 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557787    2577 flags.go:64] FLAG: --pod-cidr=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557791    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557865    2577 flags.go:64] FLAG: --pod-manifest-path=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557872    2577 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557877    2577 flags.go:64] FLAG: --pods-per-core="0"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557883    2577 flags.go:64] FLAG: --port="10250"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557888    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557893    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0363fdacef5ff37b9"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557962    2577 flags.go:64] FLAG: --qos-reserved=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557968    2577 flags.go:64] FLAG: --read-only-port="10255"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557974    2577 flags.go:64] FLAG: --register-node="true"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557984    2577 flags.go:64] FLAG: --register-with-taints=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557990    2577 flags.go:64] FLAG: --registry-burst="10"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.557995    2577 flags.go:64] FLAG: --registry-qps="5"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558000    2577 flags.go:64] FLAG: --reserved-cpus=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558005    2577 flags.go:64] FLAG: --reserved-memory=""
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558010    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558029    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558034    2577 flags.go:64] FLAG: --rotate-certificates="false"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558043    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558048    2577 flags.go:64] FLAG: --runonce="false"
Sep 14 07:47:55.565900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558053    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558058    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558063    2577 flags.go:64] FLAG: --seccomp-default="false"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558068    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558497    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558503    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558509    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558519    2577 flags.go:64] FLAG: --storage-driver-password="root"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558524    2577 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558529    2577 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558534    2577 flags.go:64] FLAG: --storage-driver-user="root"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558539    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558545    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558550    2577 flags.go:64] FLAG: --system-cgroups=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558555    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558572    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558577    2577 flags.go:64] FLAG: --tls-cert-file=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558582    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558589    2577 flags.go:64] FLAG: --tls-min-version=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558594    2577 flags.go:64] FLAG: --tls-private-key-file=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558598    2577 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558603    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558613    2577 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558618    2577 flags.go:64] FLAG: --v="2"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558624    2577 flags.go:64] FLAG: --version="false"
Sep 14 07:47:55.566528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558630    2577 flags.go:64] FLAG: --vmodule=""
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558636    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.558642    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559148    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559164    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559168    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559171    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559174    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559177    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559180    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559183    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559186    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559194    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559197    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559200    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559203    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559205    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559208    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559212    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.567158 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559215    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559218    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559221    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559223    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559226    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559228    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559231    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559234    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559236    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559239    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559242    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559245    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559247    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559250    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559252    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559255    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559258    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559261    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559264    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559267    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.567671 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559269    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559272    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559275    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559277    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559281    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559283    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559286    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559288    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559291    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559293    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559296    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559299    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559302    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559304    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559307    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559318    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559321    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559324    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559327    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559329    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.568162 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559332    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559335    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559338    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559340    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559343    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559346    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559348    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559351    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559354    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559357    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559360    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559363    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559366    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559369    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559372    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559375    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559377    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559380    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559383    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559386    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.568661 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559389    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559391    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559394    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559396    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559399    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559401    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559404    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559407    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559409    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559412    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559416    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559420    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559423    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559426    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559430    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559433    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559436    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559439    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559441    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.569189 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559445    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559448    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559454    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.559458    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.560221    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.568163    2577 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.568180    2577 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.569070    2577 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.569077    2577 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569135    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569140    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569143    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569146    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569149    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.569649 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569152    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569155    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569157    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569160    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569163    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569166    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569168    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569171    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569173    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569176    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569179    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569182    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569184    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569187    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569189    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569192    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569195    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569198    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569201    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569204    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.570017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569206    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569210    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569214    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569217    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569221    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569224    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569227    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569230    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569233    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569236    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569238    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569241    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569244    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569246    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569249    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569251    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569254    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569256    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569259    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.570498 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569261    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569264    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569267    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569269    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569272    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569275    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569277    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569280    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569283    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569285    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569288    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569290    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569293    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569296    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569298    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569301    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569304    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569307    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569310    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569313    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.571009 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569316    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569318    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569321    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569323    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569326    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569328    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569331    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569333    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569336    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569338    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569341    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569343    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569346    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569348    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569351    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569353    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569357    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569361    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569364    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569367    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.571499 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569370    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569373    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569375    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569378    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569381    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569384    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569387    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569389    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569392    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569395    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569399    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569402    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569405    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569407    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569410    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.571995 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.569415    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569531    2577 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569535    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569539    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569542    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569545    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569548    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569550    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569553    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569556    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569559    2577 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569561    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569564    2577 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569567    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569569    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569573    2577 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569577    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569580    2577 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569584    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 14 07:47:55.572365 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569587    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569591    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569594    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569597    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569599    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569602    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569605    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569608    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569610    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569613    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569615    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569618    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569621    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569623    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569626    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569628    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569631    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569634    2577 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569637    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 14 07:47:55.572826 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569641    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569644    2577 feature_gate.go:328] unrecognized feature gate: Example
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569646    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569649    2577 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569652    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569655    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569658    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569660    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569663    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569666    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569669    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569672    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569675    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569678    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569680    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569683    2577 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569685    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569688    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569690    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 14 07:47:55.573419 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569693    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569695    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569698    2577 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569700    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569703    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569705    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569709    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569711    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569714    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569716    2577 feature_gate.go:328] unrecognized feature gate: Example2
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569719    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569721    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569724    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569726    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569729    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569731    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569734    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569736    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569739    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569742    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 14 07:47:55.573872 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569744    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569747    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569749    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569753    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569755    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569758    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569760    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569763    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569765    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569768    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569770    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569773    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569776    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569778    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569781    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569783    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569786    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569788    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569791    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569794    2577 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 14 07:47:55.574366 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569797    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569799    2577 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:55.569802    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.569807    2577 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.569908    2577 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.572204    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.573339    2577 server.go:1019] "Starting client certificate rotation"
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.573437    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:47:55.574847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.573495    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 14 07:47:55.576088 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.576075    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:47:55.579047 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.579024    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 14 07:47:55.586639 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.586620    2577 log.go:25] "Validated CRI v1 runtime API"
Sep 14 07:47:55.590811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.590796    2577 log.go:25] "Validated CRI v1 image API"
Sep 14 07:47:55.591623 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.591608    2577 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 14 07:47:55.596383 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.596012    2577 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 8df9f06c-78e5-4afa-a267-5503fce03f2c:/dev/nvme0n1p3 ba0d3420-64ce-4bba-a32f-63a9b30544e0:/dev/nvme0n1p4]
Sep 14 07:47:55.596468 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.596379    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 14 07:47:55.602399 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.602287    2577 manager.go:216] Machine: {Timestamp:2026-09-14 07:47:55.600848482 +0000 UTC m=+0.375346808 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3064397 MemoryCapacity:33164468224 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e30e5ea9cd1b105376cf2611fc983 SystemUUID:ec2e30e5-ea9c-d1b1-0537-6cf2611fc983 BootID:f2a3c840-5ab5-4f7c-a965-79dd40052c24 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582234112 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:fd:69:f6:55 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:fd:69:f6:55 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7e:55:b7:9b:6a:f6 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164468224 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 14 07:47:55.602399 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.602388    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 14 07:47:55.602554 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.602465    2577 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 14 07:47:55.606118 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.606088    2577 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 14 07:47:55.606274 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.606120    2577 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 14 07:47:55.607034 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.607022    2577 topology_manager.go:138] "Creating topology manager with none policy"
Sep 14 07:47:55.607090 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.607040    2577 container_manager_linux.go:309] "Creating device plugin manager"
Sep 14 07:47:55.607090 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.607052    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:47:55.607942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.607930    2577 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 14 07:47:55.609170 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.609158    2577 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 14 07:47:55.614590 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614576    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:47:55.614717 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614706    2577 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 14 07:47:55.614786 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614741    2577 kubelet.go:486] "Attempting to sync node with API server"
Sep 14 07:47:55.614786 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614755    2577 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 14 07:47:55.614786 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614775    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 14 07:47:55.614912 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614788    2577 kubelet.go:398] "Adding apiserver pod source"
Sep 14 07:47:55.614912 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.614801    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 14 07:47:55.615203 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.615190    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:47:55.615263 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.615212    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 14 07:47:55.615783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.615763    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:47:55.617631 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.617616    2577 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 14 07:47:55.620443 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.620427    2577 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 14 07:47:55.620528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.620465    2577 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 14 07:47:55.622590 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622564    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 14 07:47:55.622669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622596    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 14 07:47:55.622669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622608    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 14 07:47:55.622669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622620    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 14 07:47:55.622669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622641    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 14 07:47:55.622669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622654    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622679    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622691    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622706    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622736    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622766    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 14 07:47:55.622815 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.622778    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 14 07:47:55.623783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.623773    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 14 07:47:55.623783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.623782    2577 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 14 07:47:55.627595 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.627579    2577 server.go:1268] "Started kubelet"
Sep 14 07:47:55.627696 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.627671    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 14 07:47:55.627696 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.627662    2577 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 14 07:47:55.627788 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.627725    2577 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 14 07:47:55.628476 ip-10-0-128-30 systemd[1]: Started Kubernetes Kubelet.
Sep 14 07:47:55.629113 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.629098    2577 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 14 07:47:55.630245 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.630108    2577 server.go:310] "Adding debug handlers to kubelet server"
Sep 14 07:47:55.634754 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.634731    2577 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 14 07:47:55.635458 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.635432    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 14 07:47:55.635587 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.635558    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 14 07:47:55.636067 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.636043    2577 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 14 07:47:55.639678 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.638541    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18d5206c1ac8a108  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-09-14 07:47:55.627553032 +0000 UTC m=+0.402051358,LastTimestamp:2026-09-14 07:47:55.627553032 +0000 UTC m=+0.402051358,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}"
Sep 14 07:47:55.640889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.640872    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 14 07:47:55.641574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.641554    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 14 07:47:55.641732 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.641713    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:55.642251 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642235    2577 factory.go:55] Registering systemd factory
Sep 14 07:47:55.642251 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642252    2577 factory.go:223] Registration of the systemd container factory successfully
Sep 14 07:47:55.642381 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642282    2577 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 14 07:47:55.642381 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642286    2577 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 14 07:47:55.642381 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642308    2577 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 14 07:47:55.642381 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642374    2577 reconstruct.go:97] "Volume reconstruction finished"
Sep 14 07:47:55.642381 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642384    2577 reconciler.go:29] "Reconciler: start to sync state"
Sep 14 07:47:55.642580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642467    2577 factory.go:153] Registering CRI-O factory
Sep 14 07:47:55.642580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642481    2577 factory.go:223] Registration of the crio container factory successfully
Sep 14 07:47:55.642580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642530    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 14 07:47:55.642580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642545    2577 factory.go:103] Registering Raw factory
Sep 14 07:47:55.642580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642555    2577 manager.go:1196] Started watching for new ooms in manager
Sep 14 07:47:55.642961 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.642946    2577 manager.go:349] Starting recovery of all containers
Sep 14 07:47:55.645087 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.644707    2577 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 14 07:47:55.653536 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.653363    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 14 07:47:55.653909 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.653893    2577 manager.go:354] Recovery completed
Sep 14 07:47:55.656190 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.656148    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 14 07:47:55.659019 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.659003    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.660853 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.660834    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.660942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.660862    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.660942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.660872    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.661491 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.661478    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 14 07:47:55.661491 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.661489    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.661576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.661489    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 14 07:47:55.661576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.661561    2577 state_mem.go:36] "Initialized new in-memory state store"
Sep 14 07:47:55.662223 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.662209    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.662294 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.662239    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.662294 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.662252    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.663706 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.663693    2577 policy_none.go:49] "None policy: Start"
Sep 14 07:47:55.663750 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.663711    2577 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 14 07:47:55.663795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.663725    2577 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 14 07:47:55.664097 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.664017    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18d5206c1cc4bcbf  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-30.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-09-14 07:47:55.660852415 +0000 UTC m=+0.435350743,LastTimestamp:2026-09-14 07:47:55.660852415 +0000 UTC m=+0.435350743,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}"
Sep 14 07:47:55.665845 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.665833    2577 policy_none.go:47] "Start"
Sep 14 07:47:55.682440 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.680814    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18d5206c1cc4f4a8  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-30.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-09-14 07:47:55.660866728 +0000 UTC m=+0.435365053,LastTimestamp:2026-09-14 07:47:55.660866728 +0000 UTC m=+0.435365053,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}"
Sep 14 07:47:55.685082 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.685064    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntrlk"
Sep 14 07:47:55.693547 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.693528    2577 manager.go:337] "Starting Device Plugin manager"
Sep 14 07:47:55.693636 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.693580    2577 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 14 07:47:55.693636 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.693593    2577 server.go:87] "Starting device plugin registration server"
Sep 14 07:47:55.693968 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.693870    2577 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-30.ec2.internal.18d5206c1cc518d5  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-30.ec2.internal,UID:ip-10-0-128-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-30.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-30.ec2.internal,},FirstTimestamp:2026-09-14 07:47:55.660875989 +0000 UTC m=+0.435374316,LastTimestamp:2026-09-14 07:47:55.660875989 +0000 UTC m=+0.435374316,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-30.ec2.internal,}"
Sep 14 07:47:55.694074 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.693953    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 14 07:47:55.694124 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.694084    2577 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 14 07:47:55.694222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.694208    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 14 07:47:55.694352 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.694343    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 14 07:47:55.694427 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.694411    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 14 07:47:55.694804 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.694663    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ntrlk"
Sep 14 07:47:55.694950 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.694909    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 14 07:47:55.695032 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.694987    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:55.753350 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.753318    2577 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 14 07:47:55.755526 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.753388    2577 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 14 07:47:55.755526 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.753412    2577 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 14 07:47:55.755526 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.753525    2577 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 14 07:47:55.755703 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.755567    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:55.794385 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.794359    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.795343 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.795326    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.795452 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.795357    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.795452 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.795372    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.795452 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.795401    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.803841 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.803745    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.803841 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.803772    2577 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-30.ec2.internal\": node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:55.829999 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.829977    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:55.854009 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.853986    2577 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"]
Sep 14 07:47:55.854102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.854048    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.854986 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.854971    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.855067 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.854997    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.855067 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.855007    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.857352 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.857340    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.857528 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.857515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.857564 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.857544    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.858180 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858165    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.858241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858189    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.858241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858202    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.858241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858219    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.858241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858235    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.858359 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.858248    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.860384 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.860369    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.860440 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.860396    2577 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 14 07:47:55.861028 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.861014    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientMemory"
Sep 14 07:47:55.861109 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.861044    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasNoDiskPressure"
Sep 14 07:47:55.861109 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.861059    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeHasSufficientPID"
Sep 14 07:47:55.880515 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.880497    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.884084 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.884069    2577 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-30.ec2.internal\" not found" node="ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.930514 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:55.930488    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:55.943869 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.943841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.943973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.943868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:55.943973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:55.943885    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f4b283df53c7982920aa7f76f6f0d43-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"6f4b283df53c7982920aa7f76f6f0d43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.031044 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.031003    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:56.044391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.044459 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044394    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.044459 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f4b283df53c7982920aa7f76f6f0d43-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"6f4b283df53c7982920aa7f76f6f0d43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.044459 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/6f4b283df53c7982920aa7f76f6f0d43-config\") pod \"kube-apiserver-proxy-ip-10-0-128-30.ec2.internal\" (UID: \"6f4b283df53c7982920aa7f76f6f0d43\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.044624 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044495    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.044624 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.044497    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8f6e01bcf753632d9d36944d56c8ca24-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal\" (UID: \"8f6e01bcf753632d9d36944d56c8ca24\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.131883 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.131809    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:56.183378 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.183344    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.186520 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.186501    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.232117 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.232085    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:56.339295 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.339243    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:56.441555 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.441468    2577 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-30.ec2.internal\" not found"
Sep 14 07:47:56.460454 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.460424    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:56.542620 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.542572    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.557825 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.557799    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:47:56.559338 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.559321    2577 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal"
Sep 14 07:47:56.569264 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.569245    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 14 07:47:56.572587 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.572568    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 14 07:47:56.572701 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.572683    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:47:56.572764 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.572723    2577 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 14 07:47:56.615469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.615443    2577 apiserver.go:52] "Watching apiserver"
Sep 14 07:47:56.636305 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.636278    2577 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:56.637854 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.637832    2577 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 14 07:47:56.638706 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.638683    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-zll4h","kube-system/global-pull-secret-syncer-77jmw","kube-system/konnectivity-agent-vwtrh","openshift-cluster-node-tuning-operator/tuned-rzzvb","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal","openshift-multus/multus-additional-cni-plugins-nwd45","openshift-network-diagnostics/network-check-target-88w4r","openshift-ovn-kubernetes/ovnkube-node-scw45","kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s","openshift-dns/node-resolver-cwmch","openshift-image-registry/node-ca-jn5w7","openshift-multus/multus-5gv2g","openshift-multus/network-metrics-daemon-knfrc"]
Sep 14 07:47:56.641306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.641291    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 14 07:47:56.641422 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.641407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.644900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.644879    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 14 07:47:56.644900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.644892    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.645137 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.645115    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mb5cd\""
Sep 14 07:47:56.645258 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.645212    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.645320 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.645279    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.647579 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.647557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-lr4kj\""
Sep 14 07:47:56.647686 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.647563    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.647759 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.647678    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.647829 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.647775    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:47:56.647829 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.647798    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.648884 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-socket-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.648930 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-sys-fs\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.648930 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648915    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fr42p\" (UniqueName: \"kubernetes.io/projected/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kube-api-access-fr42p\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.649056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ac03f35-5505-4be8-b807-f27b60d3b440-hosts-file\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.649056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.649056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-registration-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.649056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.648991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-device-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.649056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.649035    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.649225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.649061    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ac03f35-5505-4be8-b807-f27b60d3b440-tmp-dir\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.649225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.649085    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vpspz\" (UniqueName: \"kubernetes.io/projected/9ac03f35-5505-4be8-b807-f27b60d3b440-kube-api-access-vpspz\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.650236 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.650220    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.652603 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.652584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.652701 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.652647    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.652766 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.652736    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4nmtm\""
Sep 14 07:47:56.653032 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.653018    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.653306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.653292    2577 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 14 07:47:56.655182 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.655152    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.655182 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.655168    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.655350 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.655332    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-jjn58\""
Sep 14 07:47:56.655418 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.655358    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.655514 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.655497    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 14 07:47:56.657796 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.657770    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 14 07:47:56.658054 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658034    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 14 07:47:56.658152 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-fmbvn\""
Sep 14 07:47:56.658152 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658136    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.658249 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.658369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658354    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.658892 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.658877    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 14 07:47:56.660553 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.660535    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.660625 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.660604    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:47:56.660721 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.660704    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 14 07:47:56.660787 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.660742    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w922p\""
Sep 14 07:47:56.663079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.663063    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.665404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.665382    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:56.665493 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.665457    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:47:56.665732 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.665709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.665823 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.665716    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 14 07:47:56.665823 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.665769    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-562nk\""
Sep 14 07:47:56.666007 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.665989    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.667968 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.667954    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.670653 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.670634    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.670742 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.670728    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 14 07:47:56.670800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.670732    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 14 07:47:56.671028 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.671011    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 14 07:47:56.671237 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.671217    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 14 07:47:56.671332 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.671232    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dzrlc\""
Sep 14 07:47:56.671332 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.671304    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 14 07:47:56.672343 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.672328    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 14 07:47:56.673261 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.673245    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-n56s4\""
Sep 14 07:47:56.673391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.673378    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 14 07:47:56.673725 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.673709    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 14 07:47:56.681735 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.681712    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rpwr9"
Sep 14 07:47:56.682499 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.682478    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rpwr9"
Sep 14 07:47:56.700976 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.700761    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-13 07:42:56 +0000 UTC" deadline="2028-06-01 19:21:23.066566328 +0000 UTC"
Sep 14 07:47:56.701103 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.701072    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15035h33m26.365640883s"
Sep 14 07:47:56.743197 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.743170    2577 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 14 07:47:56.750122 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-kubernetes\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750232 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750128    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-tmp\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750232 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750148    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-system-cni-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.750232 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750164    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-kubelet-config\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.750232 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-env-overrides\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750230    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-hostroot\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-conf-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750297    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-socket-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750320    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bnx7q\" (UniqueName: \"kubernetes.io/projected/75740f09-a43d-494d-be3a-83ba623ef515-kube-api-access-bnx7q\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.750368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750354    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-modprobe-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-var-lib-kubelet\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750423    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-lib-modules\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6v2l\" (UniqueName: \"kubernetes.io/projected/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-kube-api-access-b6v2l\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-socket-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750465    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cnibin\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-59l2h\" (UniqueName: \"kubernetes.io/projected/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-kube-api-access-59l2h\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-dbus\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750525    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysconfig\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750540    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750567    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-k8s-cni-cncf-io\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwlt5\" (UniqueName: \"kubernetes.io/projected/0d86d587-f44a-4edb-9a39-0f33f53d3909-kube-api-access-lwlt5\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vpspz\" (UniqueName: \"kubernetes.io/projected/9ac03f35-5505-4be8-b807-f27b60d3b440-kube-api-access-vpspz\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750633    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-host\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750682    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-etc-selinux\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750703    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-cnibin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-node-log\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750763    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-daemon-config\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750779    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-script-lib\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750795    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-registration-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.750876 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750841    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-systemd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-registration-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-bin\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750929    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-multus\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-tuned\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750966    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-sys-fs\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.750989    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ac03f35-5505-4be8-b807-f27b60d3b440-hosts-file\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751037    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-sys-fs\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751077    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-netns\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751108    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9ac03f35-5505-4be8-b807-f27b60d3b440-hosts-file\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751138    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-run\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751165    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-ovn\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-conf\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc7c6845-1183-4552-9363-7deddb27a825-konnectivity-ca\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751212    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-system-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751234    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-cni-binary-copy\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-bin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751286    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-multus-certs\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17439d8e-bfe4-4240-b363-9a30fc01d5c6-iptables-alerter-script\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751329    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-netns\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751350    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ac03f35-5505-4be8-b807-f27b60d3b440-tmp-dir\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-sys\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751463    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc7c6845-1183-4552-9363-7deddb27a825-agent-certs\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad93e5d6-b44c-4e63-9745-a431667d9410-serviceca\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751503    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751518    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-socket-dir-parent\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17439d8e-bfe4-4240-b363-9a30fc01d5c6-host-slash\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9j82\" (UniqueName: \"kubernetes.io/projected/17439d8e-bfe4-4240-b363-9a30fc01d5c6-kube-api-access-b9j82\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751601    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-etc-kubernetes\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751618    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-kubelet\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-etc-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751651    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-log-socket\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.751889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751662    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ac03f35-5505-4be8-b807-f27b60d3b440-tmp-dir\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l95zv\" (UniqueName: \"kubernetes.io/projected/468850b2-f30d-49d0-aa49-7be726bc68d7-kube-api-access-l95zv\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751723    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-device-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k599f\" (UniqueName: \"kubernetes.io/projected/ad93e5d6-b44c-4e63-9745-a431667d9410-kube-api-access-k599f\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-systemd-units\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751814    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-var-lib-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751848    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751858    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-device-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751874    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-config\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751892    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75740f09-a43d-494d-be3a-83ba623ef515-ovn-node-metrics-cert\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751909    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fr42p\" (UniqueName: \"kubernetes.io/projected/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kube-api-access-fr42p\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751925    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-systemd\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751962    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.751985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad93e5d6-b44c-4e63-9745-a431667d9410-host\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.752531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-kubelet\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.753044 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-slash\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.753044 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-netd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.753044 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752129    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.753044 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.752229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.758181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.758155    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 14 07:47:56.762053 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.762035    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vpspz\" (UniqueName: \"kubernetes.io/projected/9ac03f35-5505-4be8-b807-f27b60d3b440-kube-api-access-vpspz\") pod \"node-resolver-cwmch\" (UID: \"9ac03f35-5505-4be8-b807-f27b60d3b440\") " pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.762134 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.762089    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fr42p\" (UniqueName: \"kubernetes.io/projected/908b6fe5-c134-4d66-a7e6-8f935fc26c8f-kube-api-access-fr42p\") pod \"aws-ebs-csi-driver-node-n5g6s\" (UID: \"908b6fe5-c134-4d66-a7e6-8f935fc26c8f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.852721 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.852721 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852721    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-config\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.852915 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852741    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75740f09-a43d-494d-be3a-83ba623ef515-ovn-node-metrics-cert\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.852915 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-systemd\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.852915 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.852915 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:56.852915 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad93e5d6-b44c-4e63-9745-a431667d9410-host\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.852997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-systemd\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853009    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-kubelet\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-slash\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853058    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-netd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853062    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad93e5d6-b44c-4e63-9745-a431667d9410-host\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853083    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-kubernetes\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-kubelet\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853107    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-tmp\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853114    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-slash\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-netd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853199    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-system-cni-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853226    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-kubelet-config\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853255    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-env-overrides\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-system-cni-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853261    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-kubernetes\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853293    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-hostroot\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853316    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-kubelet-config\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853330    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-hostroot\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853381    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-conf-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853397    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-config\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bnx7q\" (UniqueName: \"kubernetes.io/projected/75740f09-a43d-494d-be3a-83ba623ef515-kube-api-access-bnx7q\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853457    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-conf-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853471    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853497    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-modprobe-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-var-lib-kubelet\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.853783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-lib-modules\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6v2l\" (UniqueName: \"kubernetes.io/projected/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-kube-api-access-b6v2l\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-modprobe-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853686    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-env-overrides\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853677    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cnibin\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853720    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cnibin\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-59l2h\" (UniqueName: \"kubernetes.io/projected/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-kube-api-access-59l2h\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853756    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-dbus\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysconfig\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853790    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853806    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853833    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-k8s-cni-cncf-io\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853838    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-lib-modules\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853885    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-k8s-cni-cncf-io\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853905    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f868c088-452d-4089-bf15-4c76d7ed82a4-dbus\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853758    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-var-lib-kubelet\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.853987    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysconfig\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.854608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwlt5\" (UniqueName: \"kubernetes.io/projected/0d86d587-f44a-4edb-9a39-0f33f53d3909-kube-api-access-lwlt5\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854127    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-host\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-d\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-cnibin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854234    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-node-log\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.854279    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854298    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854330    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-daemon-config\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854338    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-cnibin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.854374    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:57.354336818 +0000 UTC m=+2.128835154 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854398    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-node-log\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-host\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854529    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-script-lib\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.854541    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854563    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-systemd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-bin\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854614    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-multus\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.855585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-tuned\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.854687    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:57.354670362 +0000 UTC m=+2.129168695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-tuning-conf-dir\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854767    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-netns\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854814    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-run\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-multus\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854838    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-ovn\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854862    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-conf\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854874    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-cni-bin\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc7c6845-1183-4552-9363-7deddb27a825-konnectivity-ca\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854905    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-system-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854908    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-daemon-config\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-systemd\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854925    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-cni-binary-copy\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.854968    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-run\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-bin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.856505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855049    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-multus-certs\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855077    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-sysctl-conf\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855074    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17439d8e-bfe4-4240-b363-9a30fc01d5c6-iptables-alerter-script\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855115    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-netns\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855134    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-ovn\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855153    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-sys\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855178    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc7c6845-1183-4552-9363-7deddb27a825-agent-certs\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855205    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad93e5d6-b44c-4e63-9745-a431667d9410-serviceca\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855280    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-system-cni-dir\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855294    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-socket-dir-parent\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855320    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17439d8e-bfe4-4240-b363-9a30fc01d5c6-host-slash\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855345    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9j82\" (UniqueName: \"kubernetes.io/projected/17439d8e-bfe4-4240-b363-9a30fc01d5c6-kube-api-access-b9j82\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-etc-kubernetes\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-kubelet\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-netns\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-etc-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855506    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.857486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-log-socket\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855613    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l95zv\" (UniqueName: \"kubernetes.io/projected/468850b2-f30d-49d0-aa49-7be726bc68d7-kube-api-access-l95zv\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855651    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k599f\" (UniqueName: \"kubernetes.io/projected/ad93e5d6-b44c-4e63-9745-a431667d9410-kube-api-access-k599f\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855741    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fc7c6845-1183-4552-9363-7deddb27a825-konnectivity-ca\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856679    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-var-lib-cni-bin\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855880    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-run-netns\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/17439d8e-bfe4-4240-b363-9a30fc01d5c6-iptables-alerter-script\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856728    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-systemd-units\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855915    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-sys\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/75740f09-a43d-494d-be3a-83ba623ef515-ovnkube-script-lib\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856761    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0d86d587-f44a-4edb-9a39-0f33f53d3909-cni-binary-copy\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856050    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856111    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-etc-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856156    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-etc-kubernetes\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855849    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/17439d8e-bfe4-4240-b363-9a30fc01d5c6-host-slash\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856240    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-multus-socket-dir-parent\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.858479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856278    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-log-socket\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-run-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856514    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/75740f09-a43d-494d-be3a-83ba623ef515-ovn-node-metrics-cert\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856615    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-binary-copy\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-var-lib-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856191    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-host-kubelet\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.855943    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/0d86d587-f44a-4edb-9a39-0f33f53d3909-host-run-multus-certs\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-tmp\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856891    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-systemd-units\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856772    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/75740f09-a43d-494d-be3a-83ba623ef515-var-lib-openvswitch\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.856978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad93e5d6-b44c-4e63-9745-a431667d9410-serviceca\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.857151    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-etc-tuned\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.857379    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.859404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.858753    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fc7c6845-1183-4552-9363-7deddb27a825-agent-certs\") pod \"konnectivity-agent-vwtrh\" (UID: \"fc7c6845-1183-4552-9363-7deddb27a825\") " pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:56.860227 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.859984    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:47:56.860227 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.860017    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:47:56.860227 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.860029    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:56.860227 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:56.860072    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:57.360056007 +0000 UTC m=+2.134554322 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:56.862126 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.862102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6v2l\" (UniqueName: \"kubernetes.io/projected/ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e-kube-api-access-b6v2l\") pod \"tuned-rzzvb\" (UID: \"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e\") " pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:56.863613 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.863576    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bnx7q\" (UniqueName: \"kubernetes.io/projected/75740f09-a43d-494d-be3a-83ba623ef515-kube-api-access-bnx7q\") pod \"ovnkube-node-scw45\" (UID: \"75740f09-a43d-494d-be3a-83ba623ef515\") " pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:56.864368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.864321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwlt5\" (UniqueName: \"kubernetes.io/projected/0d86d587-f44a-4edb-9a39-0f33f53d3909-kube-api-access-lwlt5\") pod \"multus-5gv2g\" (UID: \"0d86d587-f44a-4edb-9a39-0f33f53d3909\") " pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:56.867908 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.867884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l95zv\" (UniqueName: \"kubernetes.io/projected/468850b2-f30d-49d0-aa49-7be726bc68d7-kube-api-access-l95zv\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:56.868311 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.868287    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k599f\" (UniqueName: \"kubernetes.io/projected/ad93e5d6-b44c-4e63-9745-a431667d9410-kube-api-access-k599f\") pod \"node-ca-jn5w7\" (UID: \"ad93e5d6-b44c-4e63-9745-a431667d9410\") " pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:56.869638 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.869619    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-59l2h\" (UniqueName: \"kubernetes.io/projected/ed74e3c7-89d0-434c-800a-df7e4c2d75d7-kube-api-access-59l2h\") pod \"multus-additional-cni-plugins-nwd45\" (UID: \"ed74e3c7-89d0-434c-800a-df7e4c2d75d7\") " pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:56.869889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.869867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9j82\" (UniqueName: \"kubernetes.io/projected/17439d8e-bfe4-4240-b363-9a30fc01d5c6-kube-api-access-b9j82\") pod \"iptables-alerter-zll4h\" (UID: \"17439d8e-bfe4-4240-b363-9a30fc01d5c6\") " pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:56.891704 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:56.891655    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f4b283df53c7982920aa7f76f6f0d43.slice/crio-83238f468c951a6ed95e34e64eab77103c75ad55112e0b5d08a85849b3ea3bae WatchSource:0}: Error finding container 83238f468c951a6ed95e34e64eab77103c75ad55112e0b5d08a85849b3ea3bae: Status 404 returned error can't find the container with id 83238f468c951a6ed95e34e64eab77103c75ad55112e0b5d08a85849b3ea3bae
Sep 14 07:47:56.892934 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:56.892911    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f6e01bcf753632d9d36944d56c8ca24.slice/crio-e821e1907dcff28cb04e2535eb4451ca5073301cce7f319729208bdec96c30a5 WatchSource:0}: Error finding container e821e1907dcff28cb04e2535eb4451ca5073301cce7f319729208bdec96c30a5: Status 404 returned error can't find the container with id e821e1907dcff28cb04e2535eb4451ca5073301cce7f319729208bdec96c30a5
Sep 14 07:47:56.895376 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.895363    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:47:56.967576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.967507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s"
Sep 14 07:47:56.977906 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:56.977878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod908b6fe5_c134_4d66_a7e6_8f935fc26c8f.slice/crio-acb22997cb9e35f787e06598df4c98b06c3c8ac05c680d63b983ca463b88c41d WatchSource:0}: Error finding container acb22997cb9e35f787e06598df4c98b06c3c8ac05c680d63b983ca463b88c41d: Status 404 returned error can't find the container with id acb22997cb9e35f787e06598df4c98b06c3c8ac05c680d63b983ca463b88c41d
Sep 14 07:47:56.983564 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:56.983542    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cwmch"
Sep 14 07:47:56.991868 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:56.991843    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ac03f35_5505_4be8_b807_f27b60d3b440.slice/crio-29af5d4c1efee5c479888beaa8746cb70d71d809ab28d01f083af86afd388214 WatchSource:0}: Error finding container 29af5d4c1efee5c479888beaa8746cb70d71d809ab28d01f083af86afd388214: Status 404 returned error can't find the container with id 29af5d4c1efee5c479888beaa8746cb70d71d809ab28d01f083af86afd388214
Sep 14 07:47:57.007730 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.007709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rzzvb"
Sep 14 07:47:57.011828 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.011806    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jn5w7"
Sep 14 07:47:57.016599 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.016572    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad317e69_e6ea_4a3f_9fc0_b43cdf5dbc9e.slice/crio-51f4bf967e164eb15beba8cb74add56124e6e9ca8535891a6dd02729463c4f49 WatchSource:0}: Error finding container 51f4bf967e164eb15beba8cb74add56124e6e9ca8535891a6dd02729463c4f49: Status 404 returned error can't find the container with id 51f4bf967e164eb15beba8cb74add56124e6e9ca8535891a6dd02729463c4f49
Sep 14 07:47:57.020534 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.020511    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad93e5d6_b44c_4e63_9745_a431667d9410.slice/crio-b0011423fdab8f679cfc52f72e5222be310059f478d89b5bb3e22c9263a28780 WatchSource:0}: Error finding container b0011423fdab8f679cfc52f72e5222be310059f478d89b5bb3e22c9263a28780: Status 404 returned error can't find the container with id b0011423fdab8f679cfc52f72e5222be310059f478d89b5bb3e22c9263a28780
Sep 14 07:47:57.026880 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.026858    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-nwd45"
Sep 14 07:47:57.036577 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.036554    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded74e3c7_89d0_434c_800a_df7e4c2d75d7.slice/crio-825a712d8addd84a244e7041c471f38af234ee67c76180be81930f57178a9a89 WatchSource:0}: Error finding container 825a712d8addd84a244e7041c471f38af234ee67c76180be81930f57178a9a89: Status 404 returned error can't find the container with id 825a712d8addd84a244e7041c471f38af234ee67c76180be81930f57178a9a89
Sep 14 07:47:57.043531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.043507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-5gv2g"
Sep 14 07:47:57.045178 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.045158    2577 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:57.051775 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.051754    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d86d587_f44a_4edb_9a39_0f33f53d3909.slice/crio-44ff756a6c9e1533e8b6fb22c3d88cfcd1b60c47bce9e1cc87cd57296ba2e378 WatchSource:0}: Error finding container 44ff756a6c9e1533e8b6fb22c3d88cfcd1b60c47bce9e1cc87cd57296ba2e378: Status 404 returned error can't find the container with id 44ff756a6c9e1533e8b6fb22c3d88cfcd1b60c47bce9e1cc87cd57296ba2e378
Sep 14 07:47:57.060347 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.060329    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zll4h"
Sep 14 07:47:57.068494 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.068466    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17439d8e_bfe4_4240_b363_9a30fc01d5c6.slice/crio-e50bad6bad29bf7cbcaf9eb7bbf3488f7b1e4f7ff79ff6edcefaee47311ad652 WatchSource:0}: Error finding container e50bad6bad29bf7cbcaf9eb7bbf3488f7b1e4f7ff79ff6edcefaee47311ad652: Status 404 returned error can't find the container with id e50bad6bad29bf7cbcaf9eb7bbf3488f7b1e4f7ff79ff6edcefaee47311ad652
Sep 14 07:47:57.078302 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.078285    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:47:57.084160 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.084136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:47:57.087735 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.087714    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75740f09_a43d_494d_be3a_83ba623ef515.slice/crio-d6efcd6d50e3985712e67131bb4ad1665531244e3323cc90684f3028dd67d6af WatchSource:0}: Error finding container d6efcd6d50e3985712e67131bb4ad1665531244e3323cc90684f3028dd67d6af: Status 404 returned error can't find the container with id d6efcd6d50e3985712e67131bb4ad1665531244e3323cc90684f3028dd67d6af
Sep 14 07:47:57.093323 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:47:57.093303    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc7c6845_1183_4552_9363_7deddb27a825.slice/crio-1f36a3ad6a7827a335b79f0f5e839f753f010b6d797af3b3dafd76b117823c70 WatchSource:0}: Error finding container 1f36a3ad6a7827a335b79f0f5e839f753f010b6d797af3b3dafd76b117823c70: Status 404 returned error can't find the container with id 1f36a3ad6a7827a335b79f0f5e839f753f010b6d797af3b3dafd76b117823c70
Sep 14 07:47:57.094909 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.094885    2577 kuberuntime_manager.go:1459] "Unhandled Error" err="container konnectivity-agent start failed in pod konnectivity-agent-vwtrh_kube-system(fc7c6845-1183-4552-9363-7deddb27a825): ErrImagePull: pull QPS exceeded" logger="UnhandledError"
Sep 14 07:47:57.095011 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.094924    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ErrImagePull: \"pull QPS exceeded\"" pod="kube-system/konnectivity-agent-vwtrh" podUID="fc7c6845-1183-4552-9363-7deddb27a825"
Sep 14 07:47:57.360978 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.360863    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:57.360978 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.360923    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:57.360978 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.360978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361055    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361085    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361115    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361136    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361149    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361120    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:58.361102071 +0000 UTC m=+3.135600401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361207    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:58.361190612 +0000 UTC m=+3.135688932 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:57.361255 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.361234    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:47:58.361220995 +0000 UTC m=+3.135719314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:57.412486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.412453    2577 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:57.686791 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.686627    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:42:57 +0000 UTC" deadline="2028-03-21 07:05:42.07943425 +0000 UTC"
Sep 14 07:47:57.686791 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.686688    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13295h17m44.392751207s"
Sep 14 07:47:57.757245 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.757210    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:57.757406 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.757362    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:47:57.762940 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.762252    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vwtrh" event={"ID":"fc7c6845-1183-4552-9363-7deddb27a825","Type":"ContainerStarted","Data":"1f36a3ad6a7827a335b79f0f5e839f753f010b6d797af3b3dafd76b117823c70"}
Sep 14 07:47:57.767325 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:57.766057    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"konnectivity-agent\" with ImagePullBackOff: \"Back-off pulling image \\\"quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:cb18802deb495c8d9b0ab2c3d8c3043b73eafbad562b04b71a3717870ccdff74\\\": ErrImagePull: pull QPS exceeded\"" pod="kube-system/konnectivity-agent-vwtrh" podUID="fc7c6845-1183-4552-9363-7deddb27a825"
Sep 14 07:47:57.771086 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.770625    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"d6efcd6d50e3985712e67131bb4ad1665531244e3323cc90684f3028dd67d6af"}
Sep 14 07:47:57.776026 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.775995    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rzzvb" event={"ID":"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e","Type":"ContainerStarted","Data":"51f4bf967e164eb15beba8cb74add56124e6e9ca8535891a6dd02729463c4f49"}
Sep 14 07:47:57.780755 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.780671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s" event={"ID":"908b6fe5-c134-4d66-a7e6-8f935fc26c8f","Type":"ContainerStarted","Data":"acb22997cb9e35f787e06598df4c98b06c3c8ac05c680d63b983ca463b88c41d"}
Sep 14 07:47:57.786794 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.786724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"8f6e01bcf753632d9d36944d56c8ca24","Type":"ContainerStarted","Data":"e821e1907dcff28cb04e2535eb4451ca5073301cce7f319729208bdec96c30a5"}
Sep 14 07:47:57.791847 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.791758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zll4h" event={"ID":"17439d8e-bfe4-4240-b363-9a30fc01d5c6","Type":"ContainerStarted","Data":"e50bad6bad29bf7cbcaf9eb7bbf3488f7b1e4f7ff79ff6edcefaee47311ad652"}
Sep 14 07:47:57.802078 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.802048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5gv2g" event={"ID":"0d86d587-f44a-4edb-9a39-0f33f53d3909","Type":"ContainerStarted","Data":"44ff756a6c9e1533e8b6fb22c3d88cfcd1b60c47bce9e1cc87cd57296ba2e378"}
Sep 14 07:47:57.808203 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.808116    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerStarted","Data":"825a712d8addd84a244e7041c471f38af234ee67c76180be81930f57178a9a89"}
Sep 14 07:47:57.815813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.815784    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jn5w7" event={"ID":"ad93e5d6-b44c-4e63-9745-a431667d9410","Type":"ContainerStarted","Data":"b0011423fdab8f679cfc52f72e5222be310059f478d89b5bb3e22c9263a28780"}
Sep 14 07:47:57.818081 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.818055    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cwmch" event={"ID":"9ac03f35-5505-4be8-b807-f27b60d3b440","Type":"ContainerStarted","Data":"29af5d4c1efee5c479888beaa8746cb70d71d809ab28d01f083af86afd388214"}
Sep 14 07:47:57.822981 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:57.822916    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"6f4b283df53c7982920aa7f76f6f0d43","Type":"ContainerStarted","Data":"83238f468c951a6ed95e34e64eab77103c75ad55112e0b5d08a85849b3ea3bae"}
Sep 14 07:47:58.208773 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.208531    2577 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 14 07:47:58.371317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.371148    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:58.371317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.371213    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:58.371317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.371252    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:58.371617 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.371391    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:58.371617 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.371456    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:00.371437343 +0000 UTC m=+5.145935673 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.371962    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.371989    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.372151    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.372199    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:00.372183615 +0000 UTC m=+5.146681949 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.372283    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:58.372742 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.372338    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:00.372314488 +0000 UTC m=+5.146812818 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:47:58.687419 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.687318    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-13 07:42:57 +0000 UTC" deadline="2028-02-21 05:39:57.287570697 +0000 UTC"
Sep 14 07:47:58.687419 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.687365    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12597h51m58.600210713s"
Sep 14 07:47:58.754213 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.754180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:47:58.754402 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.754325    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:47:58.754766 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:58.754746    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:47:58.754876 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:58.754854    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:47:59.756928 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:47:59.756871    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:47:59.757413 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:47:59.757045    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:00.390745    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:00.390811    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:00.390871    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391068    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391090    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391103    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391165    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:04.391144384 +0000 UTC m=+9.165642712 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391592    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391633    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:04.391621795 +0000 UTC m=+9.166120108 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391683    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:00.391757 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.391718    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:04.391706559 +0000 UTC m=+9.166204879 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:00.754942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:00.754248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:00.754942 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.754383    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:00.754942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:00.754456    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:00.754942 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:00.754533    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:01.755079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:01.755041    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:01.755581 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:01.755203    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:02.753630 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:02.753594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:02.753824 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:02.753735    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:02.754059 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:02.753593    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:02.754183 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:02.754137    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:03.754041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:03.754005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:03.754540 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:03.754155    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:04.427667 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:04.427619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:04.427923 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:04.427701    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:04.427923 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:04.427743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:04.427923 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427845    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:04.427923 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427918    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427942    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427971    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427987    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.427947    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:12.427911916 +0000 UTC m=+17.202410248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.428032    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:12.428012727 +0000 UTC m=+17.202511052 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:04.428170 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.428053    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:12.428041582 +0000 UTC m=+17.202539899 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:04.753944 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:04.753849    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:04.753944 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:04.753870    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:04.754496 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.754028    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:04.754496 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:04.754118    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:05.756155 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:05.756113    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:05.757103 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:05.757059    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:06.754254 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:06.754215    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:06.754254 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:06.754247    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:06.754527 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:06.754371    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:06.754527 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:06.754510    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:07.754221 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:07.754179    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:07.754701 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:07.754340    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:08.753659 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:08.753624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:08.753820 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:08.753624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:08.753820 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:08.753764    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:08.753936 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:08.753838    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:09.757716 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:09.757683    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:09.758162 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:09.757823    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:10.753767 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:10.753737    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:10.753906 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:10.753736    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:10.753906 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:10.753847    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:10.754026 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:10.753927    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:11.753782 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:11.753744    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:11.754277 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:11.754011    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:12.494049 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:12.493974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:12.494049 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:12.494023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:12.494071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494136    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494166    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494184    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494200    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494211    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494212    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:28.494190824 +0000 UTC m=+33.268689157 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494252    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:28.494241763 +0000 UTC m=+33.268740080 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:12.494310 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.494262    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:28.494256561 +0000 UTC m=+33.268754873 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:12.753990 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:12.753901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:12.754498 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:12.753901    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:12.754498 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.754033    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:12.754498 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:12.754154    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:13.754092 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:13.754050    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:13.754555 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:13.754202    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:14.754632 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:14.754592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:14.755113 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:14.754723    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:14.755113 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:14.754592    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:14.755113 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:14.755082    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:15.754404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:15.754375    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:15.754540 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:15.754456    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:16.754213 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.754005    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:16.754775 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.754060    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:16.754775 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:16.754321    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:16.754775 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:16.754395    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:16.871086 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.870860    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s" event={"ID":"908b6fe5-c134-4d66-a7e6-8f935fc26c8f","Type":"ContainerStarted","Data":"c23fd6f31ad2a591d92977eccf9b3575e387cb7403240df397f82b71bc562a1d"}
Sep 14 07:48:16.872605 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.872581    2577 generic.go:356] "Generic (PLEG): container finished" podID="8f6e01bcf753632d9d36944d56c8ca24" containerID="6f2aabc2a8aa5434cf8061f3137c53edf2ba3f812e1e698049f6f90dcc579508" exitCode=0
Sep 14 07:48:16.872723 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.872644    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"8f6e01bcf753632d9d36944d56c8ca24","Type":"ContainerDied","Data":"6f2aabc2a8aa5434cf8061f3137c53edf2ba3f812e1e698049f6f90dcc579508"}
Sep 14 07:48:16.874468 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.874443    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-5gv2g" event={"ID":"0d86d587-f44a-4edb-9a39-0f33f53d3909","Type":"ContainerStarted","Data":"ebe73130f6744b8e7250a3e9edbfe011c9b9e49f895da002ac21a433f2533fe7"}
Sep 14 07:48:16.876132 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.876108    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerStarted","Data":"18bdb83f1ae1782345c00b0ead1fae675c289067a89965cd13753b7c403108f5"}
Sep 14 07:48:16.877817 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.877797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jn5w7" event={"ID":"ad93e5d6-b44c-4e63-9745-a431667d9410","Type":"ContainerStarted","Data":"b85b69e0d50ae53ae3ed86297e1cbad67fe6253c2648d849257592f3ce84223a"}
Sep 14 07:48:16.879403 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.879379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cwmch" event={"ID":"9ac03f35-5505-4be8-b807-f27b60d3b440","Type":"ContainerStarted","Data":"147286248aa0fab47f344ce7b0b78ffb6de7e922d40be2af119d09c64bd72477"}
Sep 14 07:48:16.881006 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.880983    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" event={"ID":"6f4b283df53c7982920aa7f76f6f0d43","Type":"ContainerStarted","Data":"657ac892dff59a61d2dadc51fd5f1d9b142d6b5ffef7e33a2be8090566a9e220"}
Sep 14 07:48:16.883962 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.883945    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:48:16.884331 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884308    2577 generic.go:356] "Generic (PLEG): container finished" podID="75740f09-a43d-494d-be3a-83ba623ef515" containerID="a909e67aedfa49e3e40d199f2d69a7a1b5563383b88ec14366ab009c01511bb6" exitCode=1
Sep 14 07:48:16.884433 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"d4efae06d9ac6d8918ac0f73af0a76e3413d7ec818bcc5306beb4a15ca03c94e"}
Sep 14 07:48:16.884494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884440    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"3a8fad08794f171e3cc20eb8458039f9c39494ca225b35671748d70df3e2fc95"}
Sep 14 07:48:16.884548 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884492    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"5f061ed30cb76017d5be943fe4e34b4b9ef9025734cc122112a00a89106d0f24"}
Sep 14 07:48:16.884548 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884540    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerDied","Data":"a909e67aedfa49e3e40d199f2d69a7a1b5563383b88ec14366ab009c01511bb6"}
Sep 14 07:48:16.884652 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.884589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"6dbc21f4f2eaffab7d43180abd45d02a5ec4adc9c01ffe2eaa9af2cdc3b5f81a"}
Sep 14 07:48:16.886138 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.886119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rzzvb" event={"ID":"ad317e69-e6ea-4a3f-9fc0-b43cdf5dbc9e","Type":"ContainerStarted","Data":"04c922fffdf09cd097b8817b2728bc4b4111ddedbea53f2a8d3817f4431880db"}
Sep 14 07:48:16.951293 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.951241    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-30.ec2.internal" podStartSLOduration=19.951225542 podStartE2EDuration="19.951225542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:16.928558568 +0000 UTC m=+21.703056903" watchObservedRunningTime="2026-09-14 07:48:16.951225542 +0000 UTC m=+21.725723876"
Sep 14 07:48:16.976692 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.976639    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cwmch" podStartSLOduration=2.077119792 podStartE2EDuration="20.976620248s" totalImagesPullingTime="18.899500456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:16.95140893 +0000 UTC m=+21.725907265" watchObservedRunningTime="2026-09-14 07:48:16.976620248 +0000 UTC m=+21.751118582"
Sep 14 07:48:16.976860 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:16.976779    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-5gv2g" podStartSLOduration=1.555166711 podStartE2EDuration="20.976772846s" totalImagesPullingTime="19.421606135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:16.975667145 +0000 UTC m=+21.750165474" watchObservedRunningTime="2026-09-14 07:48:16.976772846 +0000 UTC m=+21.751271182"
Sep 14 07:48:17.029595 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.029550    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rzzvb" podStartSLOduration=2.123843816 podStartE2EDuration="21.02953527s" totalImagesPullingTime="18.905691454s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:17.005229438 +0000 UTC m=+21.779727776" watchObservedRunningTime="2026-09-14 07:48:17.02953527 +0000 UTC m=+21.804033618"
Sep 14 07:48:17.073616 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.073556    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jn5w7" podStartSLOduration=2.202618935 podStartE2EDuration="21.073537008s" totalImagesPullingTime="18.870918073s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:17.029596219 +0000 UTC m=+21.804094559" watchObservedRunningTime="2026-09-14 07:48:17.073537008 +0000 UTC m=+21.848035345"
Sep 14 07:48:17.753958 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.753908    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:17.754105 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:17.754041    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:17.889434 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.889398    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vwtrh" event={"ID":"fc7c6845-1183-4552-9363-7deddb27a825","Type":"ContainerStarted","Data":"bf43c3e81874001174fd6245f87e9e58d471b8555f2a5d6e78b5d67b365bac15"}
Sep 14 07:48:17.892131 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.892114    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:48:17.892540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.892521    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"7061bc1d4ca870c4aa25e1de19dc777a4143466726ee20013eba6e4e5df4431d"}
Sep 14 07:48:17.894213 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.894196    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" event={"ID":"8f6e01bcf753632d9d36944d56c8ca24","Type":"ContainerStarted","Data":"cbd87c2135735ae4d7e386bddda528d8582c9cd867cf4e0981d112f60d38741a"}
Sep 14 07:48:17.895515 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.895498    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zll4h" event={"ID":"17439d8e-bfe4-4240-b363-9a30fc01d5c6","Type":"ContainerStarted","Data":"ab63d277a01a4b237efc6d601cefdaac230b17bc071326bbd7e87938ef809a1d"}
Sep 14 07:48:17.896855 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.896836    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="18bdb83f1ae1782345c00b0ead1fae675c289067a89965cd13753b7c403108f5" exitCode=0
Sep 14 07:48:17.896931 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.896861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"18bdb83f1ae1782345c00b0ead1fae675c289067a89965cd13753b7c403108f5"}
Sep 14 07:48:17.907291 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.907260    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vwtrh" podStartSLOduration=0.688300078 podStartE2EDuration="20.907247256s" totalImagesPullingTime="20.218947178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-09-14 07:48:17.906744144 +0000 UTC m=+22.681242621" watchObservedRunningTime="2026-09-14 07:48:17.907247256 +0000 UTC m=+22.681745610"
Sep 14 07:48:17.929672 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.929637    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-30.ec2.internal" podStartSLOduration=20.929627155 podStartE2EDuration="20.929627155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:17.929309636 +0000 UTC m=+22.703807971" watchObservedRunningTime="2026-09-14 07:48:17.929627155 +0000 UTC m=+22.704125489"
Sep 14 07:48:17.970483 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:17.970449    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zll4h" podStartSLOduration=2.118483567 podStartE2EDuration="20.970437057s" totalImagesPullingTime="18.85195349s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:17.946184514 +0000 UTC m=+22.720682845" watchObservedRunningTime="2026-09-14 07:48:17.970437057 +0000 UTC m=+22.744935393"
Sep 14 07:48:18.099020 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.099001    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 14 07:48:18.704319 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.703700    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-14T07:48:18.099017546Z","UUID":"6661565f-c170-4fcc-a282-11cc50efc0c2","Handler":null,"Name":"","Endpoint":""}
Sep 14 07:48:18.705282 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.705263    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 14 07:48:18.705282 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.705296    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 14 07:48:18.753708 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.753679    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:18.753845 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.753721    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:18.753845 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:18.753825    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:18.753949 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:18.753886    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:18.901353 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:18.901312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s" event={"ID":"908b6fe5-c134-4d66-a7e6-8f935fc26c8f","Type":"ContainerStarted","Data":"69717bd210c22a6158f6c656f371e3923174e6e702a204aff44f2a2e06807024"}
Sep 14 07:48:19.754702 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:19.754507    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:19.754908 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:19.754804    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:19.906634 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:19.906608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:48:19.907041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:19.907013    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"bb6abb73e5f27f35095b75db2e665ba46406719cb04568a3f38cbc1f90ade68d"}
Sep 14 07:48:19.908832 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:19.908811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s" event={"ID":"908b6fe5-c134-4d66-a7e6-8f935fc26c8f","Type":"ContainerStarted","Data":"1827c573e51cdf9b5a8335f3b6e63e723fb9eac6fc2c5a0f1164527c736045c4"}
Sep 14 07:48:19.929361 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:19.929318    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-n5g6s" podStartSLOduration=2.733627603 podStartE2EDuration="23.929306502s" totalImagesPullingTime="21.195678899s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:19.9291032 +0000 UTC m=+24.703601547" watchObservedRunningTime="2026-09-14 07:48:19.929306502 +0000 UTC m=+24.703804837"
Sep 14 07:48:20.754197 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:20.754160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:20.754378 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:20.754160    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:20.754378 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:20.754274    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:20.754378 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:20.754351    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:21.303693 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.303657    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:48:21.304222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.304203    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:48:21.753840 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.753805    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:21.754024 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:21.753948    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:21.917090 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.917066    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:48:21.917516 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.917486    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"ad6f1674cbde9298e71f0c76df7285376c563879d2f6e0de50e14de2caf25eb1"}
Sep 14 07:48:21.917886 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.917858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:48:21.918106 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.918089    2577 scope.go:117] "RemoveContainer" containerID="a909e67aedfa49e3e40d199f2d69a7a1b5563383b88ec14366ab009c01511bb6"
Sep 14 07:48:21.918759 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:21.918734    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vwtrh"
Sep 14 07:48:22.754049 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:22.754015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:22.754581 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:22.754015    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:22.754581 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:22.754149    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:22.754581 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:22.754240    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:23.519324 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.519012    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:23.754189 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.754166    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:23.754582 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:23.754265    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:23.772347 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.772274    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-knfrc"]
Sep 14 07:48:23.774933 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.774891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88w4r"]
Sep 14 07:48:23.775048 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.774988    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:23.775105 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:23.775063    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:23.788590 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.788565    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-77jmw"]
Sep 14 07:48:23.788684 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.788639    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:23.788750 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:23.788711    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:23.925175 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.925144    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:48:23.925579 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.925548    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" event={"ID":"75740f09-a43d-494d-be3a-83ba623ef515","Type":"ContainerStarted","Data":"9e0c73e469ac80fe3e298c616f378c8728b204ae41e2dc3665daf2a2f344e4a2"}
Sep 14 07:48:23.925898 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.925869    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:23.926010 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.925961    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:23.927613 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.927591    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="897a2dde7df6c1188e574450ac25fd687f707bf781659000350712b3242cdefb" exitCode=0
Sep 14 07:48:23.927733 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.927678    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"897a2dde7df6c1188e574450ac25fd687f707bf781659000350712b3242cdefb"}
Sep 14 07:48:23.927782 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.927755    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:23.927858 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:23.927833    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:23.942262 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.942241    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:23.942376 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.942364    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:23.962654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:23.962609    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-scw45" podStartSLOduration=8.086349004 podStartE2EDuration="26.962597312s" totalImagesPullingTime="18.876248308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:23.961269684 +0000 UTC m=+28.735768018" watchObservedRunningTime="2026-09-14 07:48:23.962597312 +0000 UTC m=+28.737095629"
Sep 14 07:48:25.754494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:25.754423    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:25.755021 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:25.754506    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:25.755021 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:25.754537    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:25.755021 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:25.754590    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:25.755021 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:25.754692    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:25.755021 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:25.754763    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:25.934839 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:25.934810    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="fa0e1516cb0392c53df4463d214c759e079b7fa75069a72ae931e1c26e37a7b3" exitCode=0
Sep 14 07:48:25.934996 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:25.934896    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"fa0e1516cb0392c53df4463d214c759e079b7fa75069a72ae931e1c26e37a7b3"}
Sep 14 07:48:27.754227 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:27.754026    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:27.754658 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:27.754033    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:27.754658 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:27.754304    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-77jmw" podUID="f868c088-452d-4089-bf15-4c76d7ed82a4"
Sep 14 07:48:27.754658 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:27.754082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:27.754658 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:27.754385    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-knfrc" podUID="468850b2-f30d-49d0-aa49-7be726bc68d7"
Sep 14 07:48:27.754658 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:27.754442    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-88w4r" podUID="fd1ba2c8-c57d-482b-9d12-6285b9797938"
Sep 14 07:48:27.941662 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:27.941580    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="0e3d61f6dad9c61ed6d3ab65e63a526646c6bc83692e1b0b8c5d0779e505a0b5" exitCode=0
Sep 14 07:48:27.941662 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:27.941637    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"0e3d61f6dad9c61ed6d3ab65e63a526646c6bc83692e1b0b8c5d0779e505a0b5"}
Sep 14 07:48:28.523102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:28.523073    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:28.523207 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:28.523114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:28.523207 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:28.523146    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:28.523270 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523229    2577 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:28.523300 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523286    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret podName:f868c088-452d-4089-bf15-4c76d7ed82a4 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:00.523272697 +0000 UTC m=+65.297771014 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret") pod "global-pull-secret-syncer-77jmw" (UID: "f868c088-452d-4089-bf15-4c76d7ed82a4") : object "kube-system"/"original-pull-secret" not registered
Sep 14 07:48:28.523335 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523230    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:28.523335 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523321    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:00.523313498 +0000 UTC m=+65.297811811 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 14 07:48:28.523335 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523235    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 14 07:48:28.523426 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523346    2577 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 14 07:48:28.523426 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523357    2577 projected.go:196] Error preparing data for projected volume kube-api-access-gtcv2 for pod openshift-network-diagnostics/network-check-target-88w4r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:28.523426 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:28.523403    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2 podName:fd1ba2c8-c57d-482b-9d12-6285b9797938 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:00.523388275 +0000 UTC m=+65.297886605 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gtcv2" (UniqueName: "kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2") pod "network-check-target-88w4r" (UID: "fd1ba2c8-c57d-482b-9d12-6285b9797938") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 14 07:48:29.044511 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.044481    2577 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-30.ec2.internal" event="NodeReady"
Sep 14 07:48:29.044948 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.044592    2577 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 14 07:48:29.117051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.117018    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:48:29.136498 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.136472    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.146685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.146661    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-mrldc"]
Sep 14 07:48:29.152749 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.152727    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 14 07:48:29.153909 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.153885    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 14 07:48:29.154225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.154202    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7wdb9\""
Sep 14 07:48:29.157247 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.157225    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 14 07:48:29.165224 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.165201    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-6jdf7"]
Sep 14 07:48:29.165331 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.165207    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.171365 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.171069    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wv9fj\""
Sep 14 07:48:29.172329 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.172305    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 14 07:48:29.174200 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.173774    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 14 07:48:29.174485 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.174298    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 14 07:48:29.174871 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.174766    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 14 07:48:29.184069 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.184047    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6jdf7"]
Sep 14 07:48:29.184180 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.184092    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrldc"]
Sep 14 07:48:29.184180 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.184114    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:48:29.184180 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.184136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.189336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.189317    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 14 07:48:29.189435 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.189410    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 14 07:48:29.189625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.189608    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7986b\""
Sep 14 07:48:29.226640 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226640 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxdk7\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226898 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226898 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226845    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.226898 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226883    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.227056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.226908    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.327976 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.327941    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328005    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dzxcs\" (UniqueName: \"kubernetes.io/projected/cae33c98-1e21-45e3-8f8e-fce2594b30f0-kube-api-access-dzxcs\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328065    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328090    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae33c98-1e21-45e3-8f8e-fce2594b30f0-tmp-dir\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.328136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328121    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328137    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2rm4\" (UniqueName: \"kubernetes.io/projected/97706785-9dd7-4198-90cd-9addb0678bc0-kube-api-access-f2rm4\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cae33c98-1e21-45e3-8f8e-fce2594b30f0-config-volume\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328200    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328225    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxdk7\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.328239    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328247    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97706785-9dd7-4198-90cd-9addb0678bc0-cert\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.328256    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.328317    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:29.828299035 +0000 UTC m=+34.602797363 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:29.328411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328311    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.328767 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.328508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.329126 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.329097    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.335756 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.335730    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.335882 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.335853    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.336119 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.336080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.336119 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.336093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.338866 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.338841    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.339449 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.339428    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxdk7\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.428892 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.428831    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2rm4\" (UniqueName: \"kubernetes.io/projected/97706785-9dd7-4198-90cd-9addb0678bc0-kube-api-access-f2rm4\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.428892 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.428879    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cae33c98-1e21-45e3-8f8e-fce2594b30f0-config-volume\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.429067 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.428912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97706785-9dd7-4198-90cd-9addb0678bc0-cert\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.429067 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.428973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.429067 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.429021    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dzxcs\" (UniqueName: \"kubernetes.io/projected/cae33c98-1e21-45e3-8f8e-fce2594b30f0-kube-api-access-dzxcs\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.429226 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.429079    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae33c98-1e21-45e3-8f8e-fce2594b30f0-tmp-dir\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.429375 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.429346    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:29.429444 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.429430    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:29.929411434 +0000 UTC m=+34.703909763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:29.429517 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.429363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cae33c98-1e21-45e3-8f8e-fce2594b30f0-tmp-dir\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.429517 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.429500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/cae33c98-1e21-45e3-8f8e-fce2594b30f0-config-volume\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.431697 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.431676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/97706785-9dd7-4198-90cd-9addb0678bc0-cert\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.439203 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.439180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dzxcs\" (UniqueName: \"kubernetes.io/projected/cae33c98-1e21-45e3-8f8e-fce2594b30f0-kube-api-access-dzxcs\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.439358 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.439326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2rm4\" (UniqueName: \"kubernetes.io/projected/97706785-9dd7-4198-90cd-9addb0678bc0-kube-api-access-f2rm4\") pod \"ingress-canary-mrldc\" (UID: \"97706785-9dd7-4198-90cd-9addb0678bc0\") " pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.476334 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.476311    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-mrldc"
Sep 14 07:48:29.754008 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.753926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:48:29.754008 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.753944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:48:29.754351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.753926    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:48:29.757363 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757326    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:48:29.757495 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757366    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:48:29.757495 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757373    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:48:29.757495 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757337    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:48:29.757656 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757601    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 14 07:48:29.757656 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.757649    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 14 07:48:29.784341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.784317    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-mrldc"]
Sep 14 07:48:29.787656 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:48:29.787628    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97706785_9dd7_4198_90cd_9addb0678bc0.slice/crio-798903f094a6b6fbaf6051b079db2393e9eb1d436d105c8f52fdf446fab2f386 WatchSource:0}: Error finding container 798903f094a6b6fbaf6051b079db2393e9eb1d436d105c8f52fdf446fab2f386: Status 404 returned error can't find the container with id 798903f094a6b6fbaf6051b079db2393e9eb1d436d105c8f52fdf446fab2f386
Sep 14 07:48:29.834957 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.834929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:29.835123 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.835103    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:29.835188 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.835125    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:29.835240 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.835201    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:30.835180859 +0000 UTC m=+35.609679173 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:29.936345 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.936310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:29.936506 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.936476    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:29.936561 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:29.936547    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:30.936527669 +0000 UTC m=+35.711025981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:29.947671 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:29.947641    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrldc" event={"ID":"97706785-9dd7-4198-90cd-9addb0678bc0","Type":"ContainerStarted","Data":"798903f094a6b6fbaf6051b079db2393e9eb1d436d105c8f52fdf446fab2f386"}
Sep 14 07:48:30.844732 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:30.844691    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:30.845299 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:30.844869    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:30.845299 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:30.844892    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:30.845299 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:30.844975    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:32.844955307 +0000 UTC m=+37.619453623 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:30.946015 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:30.945733    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:30.946160 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:30.945856    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:30.946160 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:30.946152    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:32.946135187 +0000 UTC m=+37.720633513 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:32.862009 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:32.861975    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:32.862412 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:32.862149    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:32.862412 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:32.862172    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:32.862412 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:32.862236    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:36.862219137 +0000 UTC m=+41.636717474 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:32.962429 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:32.962396    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:32.962600 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:32.962528    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:32.962600 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:32.962589    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:36.96257152 +0000 UTC m=+41.737069840 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:33.960106 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:33.960028    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="080a9e3031807eba236464f46a26b89bcf6db1eb9e0bf1842bf487467fa24245" exitCode=0
Sep 14 07:48:33.960763 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:33.960105    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"080a9e3031807eba236464f46a26b89bcf6db1eb9e0bf1842bf487467fa24245"}
Sep 14 07:48:33.961604 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:33.961581    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-mrldc" event={"ID":"97706785-9dd7-4198-90cd-9addb0678bc0","Type":"ContainerStarted","Data":"f187d53df42e9c350af5811d9ea434b335296ccb6b8fd0286b0e440482dc8821"}
Sep 14 07:48:34.012047 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:34.012001    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-mrldc" podStartSLOduration=1.172951343 podStartE2EDuration="5.011981106s" totalImagesPullingTime="3.839029763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:34.011714769 +0000 UTC m=+38.786213104" watchObservedRunningTime="2026-09-14 07:48:34.011981106 +0000 UTC m=+38.786479442"
Sep 14 07:48:34.967075 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:34.967041    2577 generic.go:356] "Generic (PLEG): container finished" podID="ed74e3c7-89d0-434c-800a-df7e4c2d75d7" containerID="4a11b5e7d452f03f3acfe887fdfcf547667bdd5cd19e2f91bb96be9484e427b4" exitCode=0
Sep 14 07:48:34.967693 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:34.967130    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerDied","Data":"4a11b5e7d452f03f3acfe887fdfcf547667bdd5cd19e2f91bb96be9484e427b4"}
Sep 14 07:48:35.972288 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:35.972254    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-nwd45" event={"ID":"ed74e3c7-89d0-434c-800a-df7e4c2d75d7","Type":"ContainerStarted","Data":"d8c21d6839fdc2819db4e29d013dffc45992c2faaa0a66347a65133858c8503b"}
Sep 14 07:48:36.008189 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:36.008106    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-nwd45" podStartSLOduration=7.644195967 podStartE2EDuration="40.008089948s" totalImagesPullingTime="32.363893981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:36.006723607 +0000 UTC m=+40.781221932" watchObservedRunningTime="2026-09-14 07:48:36.008089948 +0000 UTC m=+40.782588284"
Sep 14 07:48:36.885416 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:36.885377    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:36.885614 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:36.885509    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:36.885614 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:36.885529    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:36.885614 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:36.885591    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:44.885572824 +0000 UTC m=+49.660071145 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:36.986249 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:36.986220    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:36.986665 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:36.986359    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:36.986665 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:36.986444    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:44.986426367 +0000 UTC m=+49.760924694 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:44.938001 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:44.937949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:48:44.938363 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:44.938102    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:48:44.938363 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:44.938117    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:48:44.938363 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:44.938181    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:00.938164486 +0000 UTC m=+65.712662819 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:48:45.038389 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:45.038354    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:48:45.038567 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:45.038509    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:48:45.038609 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:45.038575    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:01.038561336 +0000 UTC m=+65.813059650 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:48:50.070483 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.070448    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5844b856db-zp4c6"]
Sep 14 07:48:50.074450 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.074432    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"]
Sep 14 07:48:50.074566 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.074549    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.078350 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.078324    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.079080 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.078965    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-q7tdg\""
Sep 14 07:48:50.079643 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.079626    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.079730 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.079686    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 14 07:48:50.079929 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.079884    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.081219 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.081194    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"]
Sep 14 07:48:50.084669 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.084652    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.087831 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.087813    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 14 07:48:50.087938 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.087905    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.088009 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.087934    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 14 07:48:50.088122 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.088058    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:48:50.088237 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.088154    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.088296 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.088268    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 14 07:48:50.088545 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.088532    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 14 07:48:50.089666 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.089647    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-5bzdw\""
Sep 14 07:48:50.089850 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.089831    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s2zws\""
Sep 14 07:48:50.091314 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.091300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 14 07:48:50.096664 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.096644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"]
Sep 14 07:48:50.098138 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.098108    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 14 07:48:50.099030 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.099013    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"]
Sep 14 07:48:50.099691 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.099674    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-zp4c6"]
Sep 14 07:48:50.171139 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171115    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-trusted-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.171272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23231f53-30d5-40c0-b1d8-41c236fcb9c6-serving-cert\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.171272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.171272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171209    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knfnt\" (UniqueName: \"kubernetes.io/projected/23231f53-30d5-40c0-b1d8-41c236fcb9c6-kube-api-access-knfnt\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.171272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171246    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15061a53-ef12-442c-a185-6f02c5fb5173-config\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.171272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171266    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-service-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.171431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171307    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jqgr5\" (UniqueName: \"kubernetes.io/projected/15061a53-ef12-442c-a185-6f02c5fb5173-kube-api-access-jqgr5\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.171431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rx9t7\" (UniqueName: \"kubernetes.io/projected/8781562c-3f71-4671-82bd-35e135c0fff1-kube-api-access-rx9t7\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.171431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171398    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15061a53-ef12-442c-a185-6f02c5fb5173-serving-cert\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.171431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171412    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-tmp\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.171546 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.171429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-snapshots\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.272752 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15061a53-ef12-442c-a185-6f02c5fb5173-config\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.272752 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272754    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-service-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.272937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272776    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jqgr5\" (UniqueName: \"kubernetes.io/projected/15061a53-ef12-442c-a185-6f02c5fb5173-kube-api-access-jqgr5\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.272937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272830    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rx9t7\" (UniqueName: \"kubernetes.io/projected/8781562c-3f71-4671-82bd-35e135c0fff1-kube-api-access-rx9t7\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.272937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272856    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15061a53-ef12-442c-a185-6f02c5fb5173-serving-cert\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.272937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272872    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-tmp\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.272937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272893    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-snapshots\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273205 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.272935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-trusted-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273205 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273185    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23231f53-30d5-40c0-b1d8-41c236fcb9c6-serving-cert\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273316 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273259    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.273316 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273306    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knfnt\" (UniqueName: \"kubernetes.io/projected/23231f53-30d5-40c0-b1d8-41c236fcb9c6-kube-api-access-knfnt\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273316 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-tmp\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273522 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:50.273427    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:48:50.273522 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273440    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/15061a53-ef12-442c-a185-6f02c5fb5173-config\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.273522 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273463    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-service-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273522 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:50.273509    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls podName:8781562c-3f71-4671-82bd-35e135c0fff1 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:50.773492239 +0000 UTC m=+55.547990576 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-cscl2" (UID: "8781562c-3f71-4671-82bd-35e135c0fff1") : secret "samples-operator-tls" not found
Sep 14 07:48:50.273698 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273640    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/23231f53-30d5-40c0-b1d8-41c236fcb9c6-snapshots\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.273846 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.273826    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/23231f53-30d5-40c0-b1d8-41c236fcb9c6-trusted-ca-bundle\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.276304 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.276275    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/15061a53-ef12-442c-a185-6f02c5fb5173-serving-cert\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.276406 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.276365    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/23231f53-30d5-40c0-b1d8-41c236fcb9c6-serving-cert\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.284279 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.284254    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jqgr5\" (UniqueName: \"kubernetes.io/projected/15061a53-ef12-442c-a185-6f02c5fb5173-kube-api-access-jqgr5\") pod \"service-ca-operator-cf7c776b7-t9x7d\" (UID: \"15061a53-ef12-442c-a185-6f02c5fb5173\") " pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.285552 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.285535    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rx9t7\" (UniqueName: \"kubernetes.io/projected/8781562c-3f71-4671-82bd-35e135c0fff1-kube-api-access-rx9t7\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.288660 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.288636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knfnt\" (UniqueName: \"kubernetes.io/projected/23231f53-30d5-40c0-b1d8-41c236fcb9c6-kube-api-access-knfnt\") pod \"insights-operator-5844b856db-zp4c6\" (UID: \"23231f53-30d5-40c0-b1d8-41c236fcb9c6\") " pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.386839 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.386782    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5844b856db-zp4c6"
Sep 14 07:48:50.392839 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.392821    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"
Sep 14 07:48:50.659000 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.658971    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5844b856db-zp4c6"]
Sep 14 07:48:50.662221 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.662196    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d"]
Sep 14 07:48:50.663554 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:48:50.663521    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23231f53_30d5_40c0_b1d8_41c236fcb9c6.slice/crio-df34400ce6a39b76281ae47d4b5822f9d61792759df61e539991ccba42c8bb22 WatchSource:0}: Error finding container df34400ce6a39b76281ae47d4b5822f9d61792759df61e539991ccba42c8bb22: Status 404 returned error can't find the container with id df34400ce6a39b76281ae47d4b5822f9d61792759df61e539991ccba42c8bb22
Sep 14 07:48:50.665251 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:48:50.665229    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15061a53_ef12_442c_a185_6f02c5fb5173.slice/crio-007f1c0c8103f8a35921d9d8ea73c6458960a79eb1c8e12b96f62469da249aae WatchSource:0}: Error finding container 007f1c0c8103f8a35921d9d8ea73c6458960a79eb1c8e12b96f62469da249aae: Status 404 returned error can't find the container with id 007f1c0c8103f8a35921d9d8ea73c6458960a79eb1c8e12b96f62469da249aae
Sep 14 07:48:50.777096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:50.777068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:50.777247 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:50.777182    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:48:50.777247 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:50.777233    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls podName:8781562c-3f71-4671-82bd-35e135c0fff1 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:51.777218784 +0000 UTC m=+56.551717110 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-cscl2" (UID: "8781562c-3f71-4671-82bd-35e135c0fff1") : secret "samples-operator-tls" not found
Sep 14 07:48:51.009686 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:51.009594    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d" event={"ID":"15061a53-ef12-442c-a185-6f02c5fb5173","Type":"ContainerStarted","Data":"007f1c0c8103f8a35921d9d8ea73c6458960a79eb1c8e12b96f62469da249aae"}
Sep 14 07:48:51.010639 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:51.010613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-zp4c6" event={"ID":"23231f53-30d5-40c0-b1d8-41c236fcb9c6","Type":"ContainerStarted","Data":"df34400ce6a39b76281ae47d4b5822f9d61792759df61e539991ccba42c8bb22"}
Sep 14 07:48:51.784422 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:51.784391    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:51.784847 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:51.784541    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:48:51.784847 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:51.784614    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls podName:8781562c-3f71-4671-82bd-35e135c0fff1 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:53.784592382 +0000 UTC m=+58.559090707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-cscl2" (UID: "8781562c-3f71-4671-82bd-35e135c0fff1") : secret "samples-operator-tls" not found
Sep 14 07:48:52.277608 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.277568    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"]
Sep 14 07:48:52.280654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.280624    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"
Sep 14 07:48:52.284935 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.284899    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-psdcg\""
Sep 14 07:48:52.295673 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.295644    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"]
Sep 14 07:48:52.388400 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.388369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlz68\" (UniqueName: \"kubernetes.io/projected/16d74601-8862-4ccd-8e7e-2812fa8aa248-kube-api-access-jlz68\") pod \"network-check-source-6f8b768455-jxzxx\" (UID: \"16d74601-8862-4ccd-8e7e-2812fa8aa248\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"
Sep 14 07:48:52.489804 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.489769    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlz68\" (UniqueName: \"kubernetes.io/projected/16d74601-8862-4ccd-8e7e-2812fa8aa248-kube-api-access-jlz68\") pod \"network-check-source-6f8b768455-jxzxx\" (UID: \"16d74601-8862-4ccd-8e7e-2812fa8aa248\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"
Sep 14 07:48:52.511422 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.511391    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlz68\" (UniqueName: \"kubernetes.io/projected/16d74601-8862-4ccd-8e7e-2812fa8aa248-kube-api-access-jlz68\") pod \"network-check-source-6f8b768455-jxzxx\" (UID: \"16d74601-8862-4ccd-8e7e-2812fa8aa248\") " pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"
Sep 14 07:48:52.592385 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:52.592304    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"
Sep 14 07:48:53.273644 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:53.273567    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx"]
Sep 14 07:48:53.276804 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:48:53.276775    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d74601_8862_4ccd_8e7e_2812fa8aa248.slice/crio-66e3839a90ccffe676ca853f0745ed62d4c30703bf7e93a8fa6526908d5e86f0 WatchSource:0}: Error finding container 66e3839a90ccffe676ca853f0745ed62d4c30703bf7e93a8fa6526908d5e86f0: Status 404 returned error can't find the container with id 66e3839a90ccffe676ca853f0745ed62d4c30703bf7e93a8fa6526908d5e86f0
Sep 14 07:48:53.798704 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:53.798673    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:53.798897 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:53.798782    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:48:53.798897 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:53.798830    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls podName:8781562c-3f71-4671-82bd-35e135c0fff1 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:57.798817086 +0000 UTC m=+62.573315399 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-cscl2" (UID: "8781562c-3f71-4671-82bd-35e135c0fff1") : secret "samples-operator-tls" not found
Sep 14 07:48:54.020843 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:54.020807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-zp4c6" event={"ID":"23231f53-30d5-40c0-b1d8-41c236fcb9c6","Type":"ContainerStarted","Data":"125762bea5b596ae62c062ed9086c72c2f57e7a618373f42f00d5174012450c1"}
Sep 14 07:48:54.022469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:54.022436    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d" event={"ID":"15061a53-ef12-442c-a185-6f02c5fb5173","Type":"ContainerStarted","Data":"de3f7998fd1f3cf3ce9ddd56835a93bb673d49b88a7d93d111e4ebb8ff9258ce"}
Sep 14 07:48:54.023641 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:54.023617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx" event={"ID":"16d74601-8862-4ccd-8e7e-2812fa8aa248","Type":"ContainerStarted","Data":"66e3839a90ccffe676ca853f0745ed62d4c30703bf7e93a8fa6526908d5e86f0"}
Sep 14 07:48:54.051561 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:54.051483    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5844b856db-zp4c6" podStartSLOduration=1.708795384 podStartE2EDuration="4.051467213s" totalImagesPullingTime="2.342671829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:54.050444496 +0000 UTC m=+58.824942834" watchObservedRunningTime="2026-09-14 07:48:54.051467213 +0000 UTC m=+58.825965550"
Sep 14 07:48:55.014895 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.014835    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d" podStartSLOduration=2.6668990040000002 podStartE2EDuration="5.014814933s" totalImagesPullingTime="2.347915929s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:54.082613064 +0000 UTC m=+58.857111411" watchObservedRunningTime="2026-09-14 07:48:55.014814933 +0000 UTC m=+59.789313314"
Sep 14 07:48:55.015486 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.015114    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-x5rtk"]
Sep 14 07:48:55.022172 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.022132    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.025676 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.025650    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 14 07:48:55.025781 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.025693    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6zzx\""
Sep 14 07:48:55.025849 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.025787    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 14 07:48:55.032716 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.032696    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x5rtk"]
Sep 14 07:48:55.110412 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.110374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4d066a0-bda0-4c44-95c4-823e1a46bd83-data-volume\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.110569 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.110426    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9zsk\" (UniqueName: \"kubernetes.io/projected/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-api-access-j9zsk\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.110569 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.110471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4d066a0-bda0-4c44-95c4-823e1a46bd83-crio-socket\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.110998 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.110666    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.110998 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.110777    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212123 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212077    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212295 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.212197    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.212295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4d066a0-bda0-4c44-95c4-823e1a46bd83-data-volume\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212444 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.212327    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.712304199 +0000 UTC m=+60.486802514 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.212444 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212366    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9zsk\" (UniqueName: \"kubernetes.io/projected/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-api-access-j9zsk\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212444 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212408    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4d066a0-bda0-4c44-95c4-823e1a46bd83-crio-socket\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212590 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212489    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c4d066a0-bda0-4c44-95c4-823e1a46bd83-crio-socket\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.212590 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.212507    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.212590 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.212583    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:55.712565334 +0000 UTC m=+60.487063663 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.212750 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.212613    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c4d066a0-bda0-4c44-95c4-823e1a46bd83-data-volume\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.221969 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.221934    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9zsk\" (UniqueName: \"kubernetes.io/projected/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-api-access-j9zsk\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.717368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.717335    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.717549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.717398    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:55.717615 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.717571    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.717661 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.717632    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.717613057 +0000 UTC m=+61.492111387 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:55.717798 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.717777    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.717906 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:55.717838    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:56.717822686 +0000 UTC m=+61.492321020 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:55.955122 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:55.955090    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-scw45"
Sep 14 07:48:56.726157 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:56.726105    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:56.726693 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:56.726188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:56.726693 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:56.726210    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.726693 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:56.726288    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.726268018 +0000 UTC m=+63.500766348 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:56.726693 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:56.726331    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:56.726693 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:56.726401    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:48:58.726385871 +0000 UTC m=+63.500884188 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:57.034539 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:57.034453    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx" event={"ID":"16d74601-8862-4ccd-8e7e-2812fa8aa248","Type":"ContainerStarted","Data":"56bcc375dc644219ccdb0c5f1881b10c015fa5df8e5043481e6d199d1011c371"}
Sep 14 07:48:57.053215 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:57.053172    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f8b768455-jxzxx" podStartSLOduration=1.424504543 podStartE2EDuration="5.053157277s" totalImagesPullingTime="3.628652734s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:48:57.052344101 +0000 UTC m=+61.826842438" watchObservedRunningTime="2026-09-14 07:48:57.053157277 +0000 UTC m=+61.827655612"
Sep 14 07:48:57.771592 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:57.771568    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cwmch_9ac03f35-5505-4be8-b807-f27b60d3b440/dns-node-resolver/0.log"
Sep 14 07:48:57.835102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:57.835072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:48:57.835271 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:57.835184    2577 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 14 07:48:57.835271 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:57.835237    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls podName:8781562c-3f71-4671-82bd-35e135c0fff1 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:05.835224396 +0000 UTC m=+70.609722709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls") pod "cluster-samples-operator-86499c89c4-cscl2" (UID: "8781562c-3f71-4671-82bd-35e135c0fff1") : secret "samples-operator-tls" not found
Sep 14 07:48:58.574931 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:58.574890    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jn5w7_ad93e5d6-b44c-4e63-9745-a431667d9410/node-ca/0.log"
Sep 14 07:48:58.741571 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:58.741526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:58.741721 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:58.741586    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:48:58.741721 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:58.741652    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:48:58.741721 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:58.741698    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:58.741721 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:58.741712    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.74169806 +0000 UTC m=+67.516196396 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : configmap "kube-rbac-proxy" not found
Sep 14 07:48:58.741867 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:48:58.741741    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:02.741729932 +0000 UTC m=+67.516228272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:48:59.572127 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:48:59.572103    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrldc_97706785-9dd7-4198-90cd-9addb0678bc0/serve-healthcheck-canary/0.log"
Sep 14 07:49:00.555908 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.555874    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:49:00.556098 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.555935    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:49:00.556098 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.555963    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:49:00.558383 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.558361    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtcv2\" (UniqueName: \"kubernetes.io/projected/fd1ba2c8-c57d-482b-9d12-6285b9797938-kube-api-access-gtcv2\") pod \"network-check-target-88w4r\" (UID: \"fd1ba2c8-c57d-482b-9d12-6285b9797938\") " pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:49:00.559068 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.559042    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 14 07:49:00.559166 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.559049    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 14 07:49:00.566671 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:00.566651    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 14 07:49:00.566789 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:00.566707    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs podName:468850b2-f30d-49d0-aa49-7be726bc68d7 nodeName:}" failed. No retries permitted until 2026-09-14 07:50:04.566692276 +0000 UTC m=+129.341190602 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs") pod "network-metrics-daemon-knfrc" (UID: "468850b2-f30d-49d0-aa49-7be726bc68d7") : secret "metrics-daemon-secret" not found
Sep 14 07:49:00.568531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.568513    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f868c088-452d-4089-bf15-4c76d7ed82a4-original-pull-secret\") pod \"global-pull-secret-syncer-77jmw\" (UID: \"f868c088-452d-4089-bf15-4c76d7ed82a4\") " pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:49:00.680684 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.680657    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qxrn8\""
Sep 14 07:49:00.684156 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.684137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-77jmw"
Sep 14 07:49:00.687799 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.687783    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:49:00.959143 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.959111    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-77jmw"]
Sep 14 07:49:00.959295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.959163    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:00.959295 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:00.959259    2577 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 14 07:49:00.959295 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:00.959271    2577 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-868688f697-qjkcx: secret "image-registry-tls" not found
Sep 14 07:49:00.959473 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:00.959322    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls podName:a6e9078b-48f9-490e-b0e8-4b9419554367 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:32.959308641 +0000 UTC m=+97.733806954 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls") pod "image-registry-868688f697-qjkcx" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367") : secret "image-registry-tls" not found
Sep 14 07:49:00.962070 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:00.962043    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf868c088_452d_4089_bf15_4c76d7ed82a4.slice/crio-21719fa92e7af60fbe64d4067b4f4ac5afcd81f7817d62c0cbae4fd2e3f7c204 WatchSource:0}: Error finding container 21719fa92e7af60fbe64d4067b4f4ac5afcd81f7817d62c0cbae4fd2e3f7c204: Status 404 returned error can't find the container with id 21719fa92e7af60fbe64d4067b4f4ac5afcd81f7817d62c0cbae4fd2e3f7c204
Sep 14 07:49:00.962207 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:00.962188    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-88w4r"]
Sep 14 07:49:00.965149 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:00.965130    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd1ba2c8_c57d_482b_9d12_6285b9797938.slice/crio-cf4b8412bf43560a9d17e5943d926c6d3ed3d5ca56ea696e1564943a7fd272d6 WatchSource:0}: Error finding container cf4b8412bf43560a9d17e5943d926c6d3ed3d5ca56ea696e1564943a7fd272d6: Status 404 returned error can't find the container with id cf4b8412bf43560a9d17e5943d926c6d3ed3d5ca56ea696e1564943a7fd272d6
Sep 14 07:49:01.046928 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.046888    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-77jmw" event={"ID":"f868c088-452d-4089-bf15-4c76d7ed82a4","Type":"ContainerStarted","Data":"21719fa92e7af60fbe64d4067b4f4ac5afcd81f7817d62c0cbae4fd2e3f7c204"}
Sep 14 07:49:01.048328 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.048304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88w4r" event={"ID":"fd1ba2c8-c57d-482b-9d12-6285b9797938","Type":"ContainerStarted","Data":"15e6d3cb626f3ee86e3e9592b3e130a86bd157b96746213a5f14fe50f2c93907"}
Sep 14 07:49:01.048419 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.048345    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-88w4r" event={"ID":"fd1ba2c8-c57d-482b-9d12-6285b9797938","Type":"ContainerStarted","Data":"cf4b8412bf43560a9d17e5943d926c6d3ed3d5ca56ea696e1564943a7fd272d6"}
Sep 14 07:49:01.048455 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.048428    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:49:01.059896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.059876    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:01.060004 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:01.059991    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 14 07:49:01.060042 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:01.060038    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls podName:cae33c98-1e21-45e3-8f8e-fce2594b30f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:33.060024558 +0000 UTC m=+97.834522871 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls") pod "dns-default-6jdf7" (UID: "cae33c98-1e21-45e3-8f8e-fce2594b30f0") : secret "dns-default-metrics-tls" not found
Sep 14 07:49:01.080326 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:01.080282    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-88w4r" podStartSLOduration=64.080268725 podStartE2EDuration="1m4.080268725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:01.079631638 +0000 UTC m=+65.854129973" watchObservedRunningTime="2026-09-14 07:49:01.080268725 +0000 UTC m=+65.854767135"
Sep 14 07:49:02.774497 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:02.774459    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:02.774912 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:02.774521    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:02.774912 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:02.774591    2577 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.774912 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:02.774654    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.774639908 +0000 UTC m=+75.549138221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : configmap "kube-rbac-proxy" not found
Sep 14 07:49:02.774912 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:02.774660    2577 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:02.774912 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:02.774714    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls podName:c4d066a0-bda0-4c44-95c4-823e1a46bd83 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:10.774697165 +0000 UTC m=+75.549195493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls") pod "insights-runtime-extractor-x5rtk" (UID: "c4d066a0-bda0-4c44-95c4-823e1a46bd83") : secret "insights-runtime-extractor-tls" not found
Sep 14 07:49:05.901649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:05.901619    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:49:05.903965 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:05.903941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8781562c-3f71-4671-82bd-35e135c0fff1-samples-operator-tls\") pod \"cluster-samples-operator-86499c89c4-cscl2\" (UID: \"8781562c-3f71-4671-82bd-35e135c0fff1\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:49:06.001283 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:06.001261    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-s2zws\""
Sep 14 07:49:06.009509 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:06.009474    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"
Sep 14 07:49:06.065357 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:06.065320    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-77jmw" event={"ID":"f868c088-452d-4089-bf15-4c76d7ed82a4","Type":"ContainerStarted","Data":"6344511594dc93a393ff81f96464c8fb204ef14032a2e6d8787247d1aada807c"}
Sep 14 07:49:06.083460 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:06.083413    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-77jmw" podStartSLOduration=64.908122755 podStartE2EDuration="1m9.083398296s" totalImagesPullingTime="4.175275541s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:47:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:06.082881828 +0000 UTC m=+70.857380175" watchObservedRunningTime="2026-09-14 07:49:06.083398296 +0000 UTC m=+70.857896631"
Sep 14 07:49:06.290582 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:06.290508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2"]
Sep 14 07:49:07.069672 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:07.069630    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2" event={"ID":"8781562c-3f71-4671-82bd-35e135c0fff1","Type":"ContainerStarted","Data":"6f052561dddbb2fa6da7a1ccb5fe108d253e385aaaa0b881c1f70362296bdf1c"}
Sep 14 07:49:09.078195 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:09.078163    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2" event={"ID":"8781562c-3f71-4671-82bd-35e135c0fff1","Type":"ContainerStarted","Data":"d7338547726a6913f1c5dbe0a4c183d857268e8f41eab14566ec40eddd001680"}
Sep 14 07:49:09.078548 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:09.078217    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2" event={"ID":"8781562c-3f71-4671-82bd-35e135c0fff1","Type":"ContainerStarted","Data":"fa2baa86dbf54d6c19d934872d1f253ce02a5b9e56a248c1b9e0c7315edb18f8"}
Sep 14 07:49:09.097526 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:09.097476    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-86499c89c4-cscl2" podStartSLOduration=17.343109073 podStartE2EDuration="19.097461923s" totalImagesPullingTime="1.75435285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:09.095775229 +0000 UTC m=+73.870273566" watchObservedRunningTime="2026-09-14 07:49:09.097461923 +0000 UTC m=+73.871960257"
Sep 14 07:49:10.842544 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.842503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:10.843037 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.842555    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:10.843102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.843070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c4d066a0-bda0-4c44-95c4-823e1a46bd83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:10.844884 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.844863    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c4d066a0-bda0-4c44-95c4-823e1a46bd83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-x5rtk\" (UID: \"c4d066a0-bda0-4c44-95c4-823e1a46bd83\") " pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:10.936681 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.936654    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k6zzx\""
Sep 14 07:49:10.943910 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:10.943883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-x5rtk"
Sep 14 07:49:11.216992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:11.216959    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-x5rtk"]
Sep 14 07:49:11.219632 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:11.219605    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4d066a0_bda0_4c44_95c4_823e1a46bd83.slice/crio-64be96df5bc0c4b7d857b5cb27efb47b253675de49cb62605ff0f1ae2a6bddf7 WatchSource:0}: Error finding container 64be96df5bc0c4b7d857b5cb27efb47b253675de49cb62605ff0f1ae2a6bddf7: Status 404 returned error can't find the container with id 64be96df5bc0c4b7d857b5cb27efb47b253675de49cb62605ff0f1ae2a6bddf7
Sep 14 07:49:12.092531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:12.092497    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5rtk" event={"ID":"c4d066a0-bda0-4c44-95c4-823e1a46bd83","Type":"ContainerStarted","Data":"23b748b62c5967dfe25cab5a504ea9a28511a3d5e8b5d872b61c15a7a03c7995"}
Sep 14 07:49:12.092865 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:12.092573    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5rtk" event={"ID":"c4d066a0-bda0-4c44-95c4-823e1a46bd83","Type":"ContainerStarted","Data":"64be96df5bc0c4b7d857b5cb27efb47b253675de49cb62605ff0f1ae2a6bddf7"}
Sep 14 07:49:13.098038 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:13.097998    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5rtk" event={"ID":"c4d066a0-bda0-4c44-95c4-823e1a46bd83","Type":"ContainerStarted","Data":"66a8035acf3d1d0f3df23bce942471814f4520dcfd006d0a9516a01f98c8aed8"}
Sep 14 07:49:14.103273 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:14.103232    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-x5rtk" event={"ID":"c4d066a0-bda0-4c44-95c4-823e1a46bd83","Type":"ContainerStarted","Data":"e38b63ac6cb5792879faf6257cd3b7d25894a19261418c2cdb08a3067532d87b"}
Sep 14 07:49:14.126174 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:14.126129    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-x5rtk" podStartSLOduration=17.581943031 podStartE2EDuration="20.126114278s" totalImagesPullingTime="2.544171247s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 07:48:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:14.125356921 +0000 UTC m=+78.899855256" watchObservedRunningTime="2026-09-14 07:49:14.126114278 +0000 UTC m=+78.900612613"
Sep 14 07:49:17.816428 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.816393    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"]
Sep 14 07:49:17.819561 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.819546    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.826097 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.826067    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="2b57ee52-137c-4caa-885b-d45315ba19f3" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.826226 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.826125    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kube-root-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" type="*v1.ConfigMap"
Sep 14 07:49:17.826592 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.826572    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-hub-kubeconfig\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" type="*v1.Secret"
Sep 14 07:49:17.827161 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.827140    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"openshift-service-ca.crt\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" type="*v1.ConfigMap"
Sep 14 07:49:17.827161 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.827141    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"managed-serviceaccount-dockercfg-h6kjl\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h6kjl\"" type="*v1.Secret"
Sep 14 07:49:17.827380 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:17.827361    2577 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"open-cluster-management-image-pull-credentials\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"open-cluster-management-agent-addon\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" type="*v1.Secret"
Sep 14 07:49:17.855246 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.855218    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"]
Sep 14 07:49:17.893344 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.893248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b57ee52-137c-4caa-885b-d45315ba19f3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.893344 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.893324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kczn\" (UniqueName: \"kubernetes.io/projected/2b57ee52-137c-4caa-885b-d45315ba19f3-kube-api-access-5kczn\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.903156 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.903134    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"]
Sep 14 07:49:17.906027 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.906012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:17.912570 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.912552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-6sbcv\""
Sep 14 07:49:17.913064 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.913049    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 14 07:49:17.913233 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.913218    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 14 07:49:17.922136 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.922116    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-76f99584f4-j9s58"]
Sep 14 07:49:17.925304 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.925288    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"]
Sep 14 07:49:17.925399 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.925388    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:17.930347 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.930331    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 14 07:49:17.930452 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.930437    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-x9mdt\""
Sep 14 07:49:17.931002 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.930983    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 14 07:49:17.940400 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.940381    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-j9s58"]
Sep 14 07:49:17.993912 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.993882    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eac7b0b4-5770-4b9d-9b66-552733b153ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:17.994076 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.994017    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b57ee52-137c-4caa-885b-d45315ba19f3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.994149 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.994105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnr4b\" (UniqueName: \"kubernetes.io/projected/fae60f99-2195-40c4-9d94-09a8bed79983-kube-api-access-vnr4b\") pod \"downloads-76f99584f4-j9s58\" (UID: \"fae60f99-2195-40c4-9d94-09a8bed79983\") " pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:17.994210 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.994159    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kczn\" (UniqueName: \"kubernetes.io/projected/2b57ee52-137c-4caa-885b-d45315ba19f3-kube-api-access-5kczn\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:17.994210 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:17.994186    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eac7b0b4-5770-4b9d-9b66-552733b153ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.095461 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.095389    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eac7b0b4-5770-4b9d-9b66-552733b153ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.095461 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.095427    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eac7b0b4-5770-4b9d-9b66-552733b153ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.095627 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.095483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnr4b\" (UniqueName: \"kubernetes.io/projected/fae60f99-2195-40c4-9d94-09a8bed79983-kube-api-access-vnr4b\") pod \"downloads-76f99584f4-j9s58\" (UID: \"fae60f99-2195-40c4-9d94-09a8bed79983\") " pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:18.096144 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.096127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/eac7b0b4-5770-4b9d-9b66-552733b153ab-nginx-conf\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.097829 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.097813    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/eac7b0b4-5770-4b9d-9b66-552733b153ab-networking-console-plugin-cert\") pod \"networking-console-plugin-5dcbc7df6b-8wn7n\" (UID: \"eac7b0b4-5770-4b9d-9b66-552733b153ab\") " pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.109057 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.109033    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnr4b\" (UniqueName: \"kubernetes.io/projected/fae60f99-2195-40c4-9d94-09a8bed79983-kube-api-access-vnr4b\") pod \"downloads-76f99584f4-j9s58\" (UID: \"fae60f99-2195-40c4-9d94-09a8bed79983\") " pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:18.214377 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.214349    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"
Sep 14 07:49:18.233275 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.233255    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:18.484761 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.484735    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n"]
Sep 14 07:49:18.487660 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:18.487634    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeac7b0b4_5770_4b9d_9b66_552733b153ab.slice/crio-27d4a00be9f308c842e9009fb7ee2735e1d7800327de900a8aafbc303b021d5c WatchSource:0}: Error finding container 27d4a00be9f308c842e9009fb7ee2735e1d7800327de900a8aafbc303b021d5c: Status 404 returned error can't find the container with id 27d4a00be9f308c842e9009fb7ee2735e1d7800327de900a8aafbc303b021d5c
Sep 14 07:49:18.510561 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.510535    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-76f99584f4-j9s58"]
Sep 14 07:49:18.513778 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:18.513753    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfae60f99_2195_40c4_9d94_09a8bed79983.slice/crio-8513f37e709226051416f188b8ecd1cd2fb32049453d825f3f4dfe1b9e6e1123 WatchSource:0}: Error finding container 8513f37e709226051416f188b8ecd1cd2fb32049453d825f3f4dfe1b9e6e1123: Status 404 returned error can't find the container with id 8513f37e709226051416f188b8ecd1cd2fb32049453d825f3f4dfe1b9e6e1123
Sep 14 07:49:18.663501 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.663471    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 14 07:49:18.760535 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.760472    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 14 07:49:18.766771 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.766747    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/2b57ee52-137c-4caa-885b-d45315ba19f3-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:18.866334 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:18.866308    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 14 07:49:19.031785 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.031710    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 14 07:49:19.040021 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.039996    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kczn\" (UniqueName: \"kubernetes.io/projected/2b57ee52-137c-4caa-885b-d45315ba19f3-kube-api-access-5kczn\") pod \"managed-serviceaccount-addon-agent-656d98557d-fk7rf\" (UID: \"2b57ee52-137c-4caa-885b-d45315ba19f3\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:19.118742 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.118712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n" event={"ID":"eac7b0b4-5770-4b9d-9b66-552733b153ab","Type":"ContainerStarted","Data":"27d4a00be9f308c842e9009fb7ee2735e1d7800327de900a8aafbc303b021d5c"}
Sep 14 07:49:19.119731 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.119712    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-j9s58" event={"ID":"fae60f99-2195-40c4-9d94-09a8bed79983","Type":"ContainerStarted","Data":"8513f37e709226051416f188b8ecd1cd2fb32049453d825f3f4dfe1b9e6e1123"}
Sep 14 07:49:19.317980 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.317754    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h6kjl\""
Sep 14 07:49:19.338682 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.338597    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"
Sep 14 07:49:19.628319 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:19.628242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf"]
Sep 14 07:49:19.775080 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:19.775048    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2b57ee52_137c_4caa_885b_d45315ba19f3.slice/crio-a87d660276a6cbac9df6b28f7d8abb569641de92456bd7d0cbff3ff1ab5d58db WatchSource:0}: Error finding container a87d660276a6cbac9df6b28f7d8abb569641de92456bd7d0cbff3ff1ab5d58db: Status 404 returned error can't find the container with id a87d660276a6cbac9df6b28f7d8abb569641de92456bd7d0cbff3ff1ab5d58db
Sep 14 07:49:20.125276 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:20.125240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n" event={"ID":"eac7b0b4-5770-4b9d-9b66-552733b153ab","Type":"ContainerStarted","Data":"058411d9d9241e231f7329808b235979234493e06a0cba8532620ad4d6479979"}
Sep 14 07:49:20.126429 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:20.126405    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf" event={"ID":"2b57ee52-137c-4caa-885b-d45315ba19f3","Type":"ContainerStarted","Data":"a87d660276a6cbac9df6b28f7d8abb569641de92456bd7d0cbff3ff1ab5d58db"}
Sep 14 07:49:20.142971 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:20.142877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-5dcbc7df6b-8wn7n" podStartSLOduration=1.812386718 podStartE2EDuration="3.142862987s" totalImagesPullingTime="1.330476269s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:20.141947946 +0000 UTC m=+84.916446283" watchObservedRunningTime="2026-09-14 07:49:20.142862987 +0000 UTC m=+84.917361349"
Sep 14 07:49:23.140585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:23.140546    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf" event={"ID":"2b57ee52-137c-4caa-885b-d45315ba19f3","Type":"ContainerStarted","Data":"5bc639937a49abad98f5a9576121b0b3c81599edab2d4927839db5c580d8dfae"}
Sep 14 07:49:23.161141 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:23.161096    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-656d98557d-fk7rf" podStartSLOduration=3.383131858 podStartE2EDuration="6.161081495s" totalImagesPullingTime="2.777949637s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:23.15932921 +0000 UTC m=+87.933827559" watchObservedRunningTime="2026-09-14 07:49:23.161081495 +0000 UTC m=+87.935579829"
Sep 14 07:49:25.042859 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.042795    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"]
Sep 14 07:49:25.046221 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.046198    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:25.049156 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.049130    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 14 07:49:25.049258 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.049133    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-9bf2g\""
Sep 14 07:49:25.055511 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.055487    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"]
Sep 14 07:49:25.155830 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.155797    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pfpvq\" (UID: \"5fbe172d-11a4-4e45-9703-940c9f3bab09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:25.256770 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.256737    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pfpvq\" (UID: \"5fbe172d-11a4-4e45-9703-940c9f3bab09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:25.256981 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:25.256870    2577 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 07:49:25.256981 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:25.256956    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates podName:5fbe172d-11a4-4e45-9703-940c9f3bab09 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:25.756936231 +0000 UTC m=+90.531434549 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-pfpvq" (UID: "5fbe172d-11a4-4e45-9703-940c9f3bab09") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 14 07:49:25.761958 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.761929    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pfpvq\" (UID: \"5fbe172d-11a4-4e45-9703-940c9f3bab09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:25.764494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.764468    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/5fbe172d-11a4-4e45-9703-940c9f3bab09-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-pfpvq\" (UID: \"5fbe172d-11a4-4e45-9703-940c9f3bab09\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:25.956541 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:25.956515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:26.242775 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:26.242746    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"]
Sep 14 07:49:26.245725 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:26.245700    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbe172d_11a4_4e45_9703_940c9f3bab09.slice/crio-7a0aaa2bdafaf6ff4c92679b282e635045778ac5b5ec5c983687b1f913d8698e WatchSource:0}: Error finding container 7a0aaa2bdafaf6ff4c92679b282e635045778ac5b5ec5c983687b1f913d8698e: Status 404 returned error can't find the container with id 7a0aaa2bdafaf6ff4c92679b282e635045778ac5b5ec5c983687b1f913d8698e
Sep 14 07:49:27.155849 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:27.155807    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq" event={"ID":"5fbe172d-11a4-4e45-9703-940c9f3bab09","Type":"ContainerStarted","Data":"7a0aaa2bdafaf6ff4c92679b282e635045778ac5b5ec5c983687b1f913d8698e"}
Sep 14 07:49:29.165324 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:29.165290    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq" event={"ID":"5fbe172d-11a4-4e45-9703-940c9f3bab09","Type":"ContainerStarted","Data":"4c8486a09fc468d187da853a0ef41696c8b7752517a179051151ff379aa75170"}
Sep 14 07:49:29.165803 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:29.165458    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:29.171009 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:29.170985    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq"
Sep 14 07:49:29.183432 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:29.183391    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-pfpvq" podStartSLOduration=1.918613331 podStartE2EDuration="4.18337622s" totalImagesPullingTime="2.264762889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:29.182202801 +0000 UTC m=+93.956701139" watchObservedRunningTime="2026-09-14 07:49:29.18337622 +0000 UTC m=+93.957874566"
Sep 14 07:49:30.100440 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.100403    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"]
Sep 14 07:49:30.102998 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.102977    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.106009 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.105984    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 14 07:49:30.107092 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.107067    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 14 07:49:30.107227 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.107208    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 14 07:49:30.107303 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.107216    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 14 07:49:30.107554 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.107434    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-2lfzh\""
Sep 14 07:49:30.107554 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.107447    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 14 07:49:30.113554 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.113532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"]
Sep 14 07:49:30.198348 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.198310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llz7j\" (UniqueName: \"kubernetes.io/projected/64445ef6-a979-45ac-a513-a06ec6b5dd31-kube-api-access-llz7j\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.198773 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.198372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.198773 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.198439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.198773 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.198574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64445ef6-a979-45ac-a513-a06ec6b5dd31-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.299225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.299189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-llz7j\" (UniqueName: \"kubernetes.io/projected/64445ef6-a979-45ac-a513-a06ec6b5dd31-kube-api-access-llz7j\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.299399 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.299267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.299399 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.299312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.299577 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.299550    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64445ef6-a979-45ac-a513-a06ec6b5dd31-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.300235 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.300190    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64445ef6-a979-45ac-a513-a06ec6b5dd31-metrics-client-ca\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.301987 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.301946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-tls\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.302096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.302043    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64445ef6-a979-45ac-a513-a06ec6b5dd31-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.308558 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.308539    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-llz7j\" (UniqueName: \"kubernetes.io/projected/64445ef6-a979-45ac-a513-a06ec6b5dd31-kube-api-access-llz7j\") pod \"prometheus-operator-68bcd7f987-vpf27\" (UID: \"64445ef6-a979-45ac-a513-a06ec6b5dd31\") " pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:30.412491 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:30.412459    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"
Sep 14 07:49:31.518370 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.518336    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:49:31.520400 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.520370    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.524335 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524311    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 14 07:49:31.524467 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524343    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dfmfg\""
Sep 14 07:49:31.524467 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524348    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 14 07:49:31.524467 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524377    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 14 07:49:31.524467 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524426    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 14 07:49:31.524685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.524631    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 14 07:49:31.532695 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.532673    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:49:31.611837 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.611793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.612041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.611884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kssxl\" (UniqueName: \"kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.612041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.611943    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.612041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.611968    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.612041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.612009    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.612041 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.612045    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713243 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713212    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kssxl\" (UniqueName: \"kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713441 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713258    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713441 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713292    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713441 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713321    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713441 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.713441 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.713405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.714186 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.714157    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.714319 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.714167    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.714319 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.714228    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.716424 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.716402    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.716533 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.716449    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.723879 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.723857    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kssxl\" (UniqueName: \"kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl\") pod \"console-6c577b694f-n92h8\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") " pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:31.832658 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:31.832572    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:32.054422 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:32.054392    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-88w4r"
Sep 14 07:49:33.028565 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.028520    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:33.031313 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.031290    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"image-registry-868688f697-qjkcx\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") " pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:33.051810 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.051779    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7wdb9\""
Sep 14 07:49:33.058476 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.058444    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:33.129831 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.129803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:33.132313 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.132284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/cae33c98-1e21-45e3-8f8e-fce2594b30f0-metrics-tls\") pod \"dns-default-6jdf7\" (UID: \"cae33c98-1e21-45e3-8f8e-fce2594b30f0\") " pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:33.398379 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.398292    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7986b\""
Sep 14 07:49:33.404965 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:33.404936    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:36.194813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.194733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-76f99584f4-j9s58" event={"ID":"fae60f99-2195-40c4-9d94-09a8bed79983","Type":"ContainerStarted","Data":"19a45f038c27b17a802910ac675802cb6dc925c1c1019dfee5071d5d86a2912b"}
Sep 14 07:49:36.195150 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.195030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:36.196327 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.196300    2577 patch_prober.go:28] interesting pod/downloads-76f99584f4-j9s58 container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.132.0.16:8080/\": dial tcp 10.132.0.16:8080: connect: connection refused" start-of-body=
Sep 14 07:49:36.196409 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.196349    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-76f99584f4-j9s58" podUID="fae60f99-2195-40c4-9d94-09a8bed79983" containerName="download-server" probeResult="failure" output="Get \"http://10.132.0.16:8080/\": dial tcp 10.132.0.16:8080: connect: connection refused"
Sep 14 07:49:36.220580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.220528    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-76f99584f4-j9s58" podStartSLOduration=1.600252097 podStartE2EDuration="19.220515417s" totalImagesPullingTime="17.62026332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:36.218812085 +0000 UTC m=+100.993310419" watchObservedRunningTime="2026-09-14 07:49:36.220515417 +0000 UTC m=+100.995013752"
Sep 14 07:49:36.225338 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.225313    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:49:36.226943 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:36.226904    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e9078b_48f9_490e_b0e8_4b9419554367.slice/crio-5fcf961aaa3c675f12ff5e19431d6c6d107bc2de7196a27e64d96660fe1ee93a WatchSource:0}: Error finding container 5fcf961aaa3c675f12ff5e19431d6c6d107bc2de7196a27e64d96660fe1ee93a: Status 404 returned error can't find the container with id 5fcf961aaa3c675f12ff5e19431d6c6d107bc2de7196a27e64d96660fe1ee93a
Sep 14 07:49:36.231052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.230990    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:49:36.232879 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.232825    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-68bcd7f987-vpf27"]
Sep 14 07:49:36.234301 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:36.234279    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-6jdf7"]
Sep 14 07:49:36.235674 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:36.235654    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64445ef6_a979_45ac_a513_a06ec6b5dd31.slice/crio-e2cbd92658499d0fb406cf1ad482dc006e20abde85bc5e71a763444a7b2c293c WatchSource:0}: Error finding container e2cbd92658499d0fb406cf1ad482dc006e20abde85bc5e71a763444a7b2c293c: Status 404 returned error can't find the container with id e2cbd92658499d0fb406cf1ad482dc006e20abde85bc5e71a763444a7b2c293c
Sep 14 07:49:36.236682 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:36.236582    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7e9de41_1627_4cb6_b66b_fc09e5255352.slice/crio-2dd58ae75915e7c36f66552a767c9eb6c8cf71a212e5252298d8025392a13861 WatchSource:0}: Error finding container 2dd58ae75915e7c36f66552a767c9eb6c8cf71a212e5252298d8025392a13861: Status 404 returned error can't find the container with id 2dd58ae75915e7c36f66552a767c9eb6c8cf71a212e5252298d8025392a13861
Sep 14 07:49:36.237662 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:36.237631    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcae33c98_1e21_45e3_8f8e_fce2594b30f0.slice/crio-670f1d948a11472e51f7de52fece42a8dfe2aba551d716d66f3bb5fd271fcbf0 WatchSource:0}: Error finding container 670f1d948a11472e51f7de52fece42a8dfe2aba551d716d66f3bb5fd271fcbf0: Status 404 returned error can't find the container with id 670f1d948a11472e51f7de52fece42a8dfe2aba551d716d66f3bb5fd271fcbf0
Sep 14 07:49:37.202626 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.202567    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27" event={"ID":"64445ef6-a979-45ac-a513-a06ec6b5dd31","Type":"ContainerStarted","Data":"e2cbd92658499d0fb406cf1ad482dc006e20abde85bc5e71a763444a7b2c293c"}
Sep 14 07:49:37.204311 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.204202    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c577b694f-n92h8" event={"ID":"d7e9de41-1627-4cb6-b66b-fc09e5255352","Type":"ContainerStarted","Data":"2dd58ae75915e7c36f66552a767c9eb6c8cf71a212e5252298d8025392a13861"}
Sep 14 07:49:37.205980 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.205952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868688f697-qjkcx" event={"ID":"a6e9078b-48f9-490e-b0e8-4b9419554367","Type":"ContainerStarted","Data":"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"}
Sep 14 07:49:37.206097 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.206003    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868688f697-qjkcx" event={"ID":"a6e9078b-48f9-490e-b0e8-4b9419554367","Type":"ContainerStarted","Data":"5fcf961aaa3c675f12ff5e19431d6c6d107bc2de7196a27e64d96660fe1ee93a"}
Sep 14 07:49:37.206781 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.206713    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:37.210077 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.210047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6jdf7" event={"ID":"cae33c98-1e21-45e3-8f8e-fce2594b30f0","Type":"ContainerStarted","Data":"670f1d948a11472e51f7de52fece42a8dfe2aba551d716d66f3bb5fd271fcbf0"}
Sep 14 07:49:37.221602 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.221577    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-76f99584f4-j9s58"
Sep 14 07:49:37.231001 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.229863    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-868688f697-qjkcx" podStartSLOduration=80.229844814 podStartE2EDuration="1m20.229844814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:37.228013621 +0000 UTC m=+102.002511959" watchObservedRunningTime="2026-09-14 07:49:37.229844814 +0000 UTC m=+102.004343150"
Sep 14 07:49:37.256910 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.256493    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:49:37.277808 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.277743    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.281548 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.281522    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:49:37.292619 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.292592    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474707    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474793    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.474852 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfg8x\" (UniqueName: \"kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.475291 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.474881    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576540    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576574    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfg8x\" (UniqueName: \"kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576674    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576719    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577062 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.576743    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.577609 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.577587    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.580543 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.578568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.580543 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.579363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.580543 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.579827    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.583414 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.583306    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.586287 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.586244    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.595098 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.594988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfg8x\" (UniqueName: \"kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x\") pod \"console-584c7bf566-p8rhj\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") " pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.601419 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.601187    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:37.936411 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:37.936343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:49:39.218278 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:39.218237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584c7bf566-p8rhj" event={"ID":"2dfe099f-df73-4cdc-8454-e93bd1890451","Type":"ContainerStarted","Data":"3a515eedbf3488fb038b6d19cdc36c9d8e55ed1105925ca216dc532f992d0a06"}
Sep 14 07:49:41.422956 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:41.422855    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:49:42.233795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.233736    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c577b694f-n92h8" event={"ID":"d7e9de41-1627-4cb6-b66b-fc09e5255352","Type":"ContainerStarted","Data":"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"}
Sep 14 07:49:42.235953 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.235898    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6jdf7" event={"ID":"cae33c98-1e21-45e3-8f8e-fce2594b30f0","Type":"ContainerStarted","Data":"b2d6f0bbe30e39bcb97b4d08d1fc8fe9b0e67fe1774942c2e9b48add40089ba4"}
Sep 14 07:49:42.236083 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.235973    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-6jdf7" event={"ID":"cae33c98-1e21-45e3-8f8e-fce2594b30f0","Type":"ContainerStarted","Data":"87f8ef9feae654e65de27dbed824a366ff8fb3adb6c9d9376d998a2f4a425424"}
Sep 14 07:49:42.236083 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.236002    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:42.238080 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.238054    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27" event={"ID":"64445ef6-a979-45ac-a513-a06ec6b5dd31","Type":"ContainerStarted","Data":"7ae44f8f5d46270d1c086586cfea14d34c26a74cdaff4870ada783a4dd333560"}
Sep 14 07:49:42.238198 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.238102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27" event={"ID":"64445ef6-a979-45ac-a513-a06ec6b5dd31","Type":"ContainerStarted","Data":"45aa1c78b8fb4b041d0ee49201390f67e4f7626e4d25ff29afc966a9a2d44b36"}
Sep 14 07:49:42.239757 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.239723    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584c7bf566-p8rhj" event={"ID":"2dfe099f-df73-4cdc-8454-e93bd1890451","Type":"ContainerStarted","Data":"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"}
Sep 14 07:49:42.263135 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.263087    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c577b694f-n92h8" podStartSLOduration=6.171494772 podStartE2EDuration="11.263073547s" totalImagesPullingTime="5.091578775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.26137011 +0000 UTC m=+107.035868445" watchObservedRunningTime="2026-09-14 07:49:42.263073547 +0000 UTC m=+107.037571882"
Sep 14 07:49:42.280549 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.280501    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-584c7bf566-p8rhj" podStartSLOduration=2.462913458 podStartE2EDuration="5.280487589s" totalImagesPullingTime="2.817574131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.279849729 +0000 UTC m=+107.054348064" watchObservedRunningTime="2026-09-14 07:49:42.280487589 +0000 UTC m=+107.054985926"
Sep 14 07:49:42.299257 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.299218    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-68bcd7f987-vpf27" podStartSLOduration=7.207947048 podStartE2EDuration="12.299205028s" totalImagesPullingTime="5.09125798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.29814886 +0000 UTC m=+107.072647196" watchObservedRunningTime="2026-09-14 07:49:42.299205028 +0000 UTC m=+107.073703362"
Sep 14 07:49:42.322096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:42.322053    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-6jdf7" podStartSLOduration=68.230586676 podStartE2EDuration="1m13.322039973s" totalImagesPullingTime="5.091453297s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:42.320729501 +0000 UTC m=+107.095227836" watchObservedRunningTime="2026-09-14 07:49:42.322039973 +0000 UTC m=+107.096538308"
Sep 14 07:49:44.454157 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.454121    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jvdm8"]
Sep 14 07:49:44.481016 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.480985    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.483796 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.483770    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 14 07:49:44.483962 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.483797    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 14 07:49:44.483962 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.483832    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6jx2x\""
Sep 14 07:49:44.483962 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.483893    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 14 07:49:44.539705 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539670    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.539865 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.539865 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539743    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.539865 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539821    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-metrics-client-ca\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.540052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539862    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-textfile\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.540052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-sys\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.540052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.539976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ztvg\" (UniqueName: \"kubernetes.io/projected/e4c95dba-db92-4922-8078-b7e25d11f8f0-kube-api-access-5ztvg\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.540052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.540007    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-root\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.540052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.540030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-wtmp\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641395 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-metrics-client-ca\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641568 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641401    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-textfile\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641568 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641447    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-sys\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641568 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ztvg\" (UniqueName: \"kubernetes.io/projected/e4c95dba-db92-4922-8078-b7e25d11f8f0-kube-api-access-5ztvg\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641568 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641505    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-root\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641568 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-wtmp\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641574    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-sys\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641594    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-root\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641602    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641661    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641704    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.641811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.641738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-wtmp\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.642158 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:44.641866    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 14 07:49:44.642158 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:49:44.641993    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls podName:e4c95dba-db92-4922-8078-b7e25d11f8f0 nodeName:}" failed. No retries permitted until 2026-09-14 07:49:45.141969426 +0000 UTC m=+109.916467753 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls") pod "node-exporter-jvdm8" (UID: "e4c95dba-db92-4922-8078-b7e25d11f8f0") : secret "node-exporter-tls" not found
Sep 14 07:49:44.642158 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.642093    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-textfile\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.642360 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.642342    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-accelerators-collector-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.642665 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.642649    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e4c95dba-db92-4922-8078-b7e25d11f8f0-metrics-client-ca\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.644542 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.644515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:44.650842 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:44.650815    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ztvg\" (UniqueName: \"kubernetes.io/projected/e4c95dba-db92-4922-8078-b7e25d11f8f0-kube-api-access-5ztvg\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:45.146043 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:45.146008    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:45.148214 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:45.148188    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e4c95dba-db92-4922-8078-b7e25d11f8f0-node-exporter-tls\") pod \"node-exporter-jvdm8\" (UID: \"e4c95dba-db92-4922-8078-b7e25d11f8f0\") " pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:45.392121 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:45.392064    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jvdm8"
Sep 14 07:49:45.431807 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:45.431781    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4c95dba_db92_4922_8078_b7e25d11f8f0.slice/crio-52e87413b24da890aa041b039ba50820f87d09baea7a004626a0476f9161798b WatchSource:0}: Error finding container 52e87413b24da890aa041b039ba50820f87d09baea7a004626a0476f9161798b: Status 404 returned error can't find the container with id 52e87413b24da890aa041b039ba50820f87d09baea7a004626a0476f9161798b
Sep 14 07:49:46.262525 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:46.262491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvdm8" event={"ID":"e4c95dba-db92-4922-8078-b7e25d11f8f0","Type":"ContainerStarted","Data":"52e87413b24da890aa041b039ba50820f87d09baea7a004626a0476f9161798b"}
Sep 14 07:49:47.270290 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:47.270257    2577 generic.go:356] "Generic (PLEG): container finished" podID="e4c95dba-db92-4922-8078-b7e25d11f8f0" containerID="c5485cd6d3f8e9e3588529fff7cd7a4acb62c1edb23d1dda2001d558d00dab55" exitCode=0
Sep 14 07:49:47.270627 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:47.270304    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvdm8" event={"ID":"e4c95dba-db92-4922-8078-b7e25d11f8f0","Type":"ContainerDied","Data":"c5485cd6d3f8e9e3588529fff7cd7a4acb62c1edb23d1dda2001d558d00dab55"}
Sep 14 07:49:47.602315 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:47.602237    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:47.602487 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:47.602327    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:47.607373 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:47.607137    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:48.276079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:48.276041    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvdm8" event={"ID":"e4c95dba-db92-4922-8078-b7e25d11f8f0","Type":"ContainerStarted","Data":"5e76ddf9b9e4e9a4976bae2ccaf6102b1a4701f3a4573c2048bf06f9d1b90bc8"}
Sep 14 07:49:48.276649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:48.276102    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jvdm8" event={"ID":"e4c95dba-db92-4922-8078-b7e25d11f8f0","Type":"ContainerStarted","Data":"71a7d7f92e919afa0e94d6d3c750db4b5b7423af7b90ff9a70f26463aefcb601"}
Sep 14 07:49:48.280105 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:48.280080    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:49:48.299958 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:48.299899    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jvdm8" podStartSLOduration=2.599712832 podStartE2EDuration="4.299886339s" totalImagesPullingTime="1.62449276s" totalInitContainerRuntime="75.680747ms" isStatefulPod=true podCreationTimestamp="2026-09-14 07:49:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:48.298136213 +0000 UTC m=+113.072634548" watchObservedRunningTime="2026-09-14 07:49:48.299886339 +0000 UTC m=+113.074384687"
Sep 14 07:49:48.364884 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:48.364851    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:49:49.677763 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.677729    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6d59cbb684-tz99m"]
Sep 14 07:49:49.693829 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.693803    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.694286 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.694263    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d59cbb684-tz99m"]
Sep 14 07:49:49.696470 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.696443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 14 07:49:49.696470 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.696460    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 14 07:49:49.696625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.696443    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 14 07:49:49.696625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.696594    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 14 07:49:49.696872 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.696852    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-jcqk6\""
Sep 14 07:49:49.697056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.697037    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 14 07:49:49.702034 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.702003    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 14 07:49:49.791786 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791760    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.791973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791801    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-serving-certs-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.791973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791820    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.791973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-metrics-client-ca\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.791973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.791973 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.791940    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gsklf\" (UniqueName: \"kubernetes.io/projected/d5c2c9f5-834e-481c-83b5-5791f22bff99-kube-api-access-gsklf\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.792165 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.792019    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-federate-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.792165 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.792043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893248 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893270    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893312    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-serving-certs-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893404 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-metrics-client-ca\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893451    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893490    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gsklf\" (UniqueName: \"kubernetes.io/projected/d5c2c9f5-834e-481c-83b5-5791f22bff99-kube-api-access-gsklf\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.893574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.893567    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-federate-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.894228 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.894175    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-metrics-client-ca\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.894360 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.894334    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.894489 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.894464    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5c2c9f5-834e-481c-83b5-5791f22bff99-serving-certs-ca-bundle\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.896014 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.895990    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-telemeter-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.896014 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.896004    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-federate-client-tls\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.896179 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.896162    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.896349 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.896331    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d5c2c9f5-834e-481c-83b5-5791f22bff99-secret-telemeter-client\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:49.902444 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:49.902417    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gsklf\" (UniqueName: \"kubernetes.io/projected/d5c2c9f5-834e-481c-83b5-5791f22bff99-kube-api-access-gsklf\") pod \"telemeter-client-6d59cbb684-tz99m\" (UID: \"d5c2c9f5-834e-481c-83b5-5791f22bff99\") " pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:50.004539 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:50.004457    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m"
Sep 14 07:49:50.303881 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:50.303812    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6d59cbb684-tz99m"]
Sep 14 07:49:50.306595 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:50.306565    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5c2c9f5_834e_481c_83b5_5791f22bff99.slice/crio-b8d571787504b76c9f88a9f6fe729cd8638bd8cca5d9db08bf08d55a43eb6375 WatchSource:0}: Error finding container b8d571787504b76c9f88a9f6fe729cd8638bd8cca5d9db08bf08d55a43eb6375: Status 404 returned error can't find the container with id b8d571787504b76c9f88a9f6fe729cd8638bd8cca5d9db08bf08d55a43eb6375
Sep 14 07:49:51.290324 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:51.290286    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m" event={"ID":"d5c2c9f5-834e-481c-83b5-5791f22bff99","Type":"ContainerStarted","Data":"b8d571787504b76c9f88a9f6fe729cd8638bd8cca5d9db08bf08d55a43eb6375"}
Sep 14 07:49:51.445500 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:51.445464    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:49:51.832863 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:51.832828    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:49:52.246678 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:52.246652    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-6jdf7"
Sep 14 07:49:53.300711 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:53.300671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m" event={"ID":"d5c2c9f5-834e-481c-83b5-5791f22bff99","Type":"ContainerStarted","Data":"c458d1080627ea8216bfe12bfca3497ed3b30a8faddf60fb7688a73b94b92940"}
Sep 14 07:49:54.306886 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:54.306863    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m" event={"ID":"d5c2c9f5-834e-481c-83b5-5791f22bff99","Type":"ContainerStarted","Data":"6d69984c1376fc030695e09cff0c1e37dae9c22d4c4164341a3892436b4dfead"}
Sep 14 07:49:55.312761 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.312726    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m" event={"ID":"d5c2c9f5-834e-481c-83b5-5791f22bff99","Type":"ContainerStarted","Data":"55260f556de31390713e48f9b32d75adbe413f4c9cc40e77e308437e26ad03d4"}
Sep 14 07:49:55.346318 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.346273    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6d59cbb684-tz99m" podStartSLOduration=2.679733897 podStartE2EDuration="6.346252202s" totalImagesPullingTime="3.666518305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:49 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:55.344159908 +0000 UTC m=+120.118658246" watchObservedRunningTime="2026-09-14 07:49:55.346252202 +0000 UTC m=+120.120750539"
Sep 14 07:49:55.525662 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.525631    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:49:55.546356 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.546316    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:49:55.546508 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.546468    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643407 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643326    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t5qm\" (UniqueName: \"kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643407 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643407 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643400    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643616 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643616 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643616 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643566    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.643616 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.643595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744286 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744286 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744290    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744524 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744524 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744524 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744413    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744524 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744436    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t5qm\" (UniqueName: \"kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.744524 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.744475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.745224 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.745195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.745368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.745195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.745368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.745271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.745653 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.745631    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.746867 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.746842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.747384 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.747363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.755226 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.755198    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t5qm\" (UniqueName: \"kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm\") pod \"console-b545c7558-drgw7\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") " pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:55.857013 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:55.856972    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:49:56.173106 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:56.173076    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:49:56.175975 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:49:56.175947    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a65ef9d_c056_48ee_85bf_6debed7dc45d.slice/crio-c86392bbc4f4ddf052001ddf460fe76d0aeb947d0d92ff3a9cf4d00e52dc496a WatchSource:0}: Error finding container c86392bbc4f4ddf052001ddf460fe76d0aeb947d0d92ff3a9cf4d00e52dc496a: Status 404 returned error can't find the container with id c86392bbc4f4ddf052001ddf460fe76d0aeb947d0d92ff3a9cf4d00e52dc496a
Sep 14 07:49:56.320100 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:56.320049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b545c7558-drgw7" event={"ID":"4a65ef9d-c056-48ee-85bf-6debed7dc45d","Type":"ContainerStarted","Data":"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"}
Sep 14 07:49:56.320437 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:56.320119    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b545c7558-drgw7" event={"ID":"4a65ef9d-c056-48ee-85bf-6debed7dc45d","Type":"ContainerStarted","Data":"c86392bbc4f4ddf052001ddf460fe76d0aeb947d0d92ff3a9cf4d00e52dc496a"}
Sep 14 07:49:56.355111 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:49:56.355061    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b545c7558-drgw7" podStartSLOduration=1.355044803 podStartE2EDuration="1.355044803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:49:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:49:56.353938214 +0000 UTC m=+121.128436540" watchObservedRunningTime="2026-09-14 07:49:56.355044803 +0000 UTC m=+121.129543201"
Sep 14 07:50:04.622112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:04.622072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:50:04.624341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:04.624319    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468850b2-f30d-49d0-aa49-7be726bc68d7-metrics-certs\") pod \"network-metrics-daemon-knfrc\" (UID: \"468850b2-f30d-49d0-aa49-7be726bc68d7\") " pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:50:04.871164 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:04.871132    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bd4sd\""
Sep 14 07:50:04.879282 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:04.879213    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-knfrc"
Sep 14 07:50:05.148373 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:05.148339    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-knfrc"]
Sep 14 07:50:05.151017 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:50:05.150983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468850b2_f30d_49d0_aa49_7be726bc68d7.slice/crio-90a5afa76bc0d221dac909b839dfa2994726cd35bb21bccc1fb7e94c7c63cebe WatchSource:0}: Error finding container 90a5afa76bc0d221dac909b839dfa2994726cd35bb21bccc1fb7e94c7c63cebe: Status 404 returned error can't find the container with id 90a5afa76bc0d221dac909b839dfa2994726cd35bb21bccc1fb7e94c7c63cebe
Sep 14 07:50:05.354273 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:05.354240    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knfrc" event={"ID":"468850b2-f30d-49d0-aa49-7be726bc68d7","Type":"ContainerStarted","Data":"90a5afa76bc0d221dac909b839dfa2994726cd35bb21bccc1fb7e94c7c63cebe"}
Sep 14 07:50:05.857232 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:05.857197    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:50:05.857661 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:05.857256    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:50:05.861879 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:05.861858    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:50:06.360338 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.360312    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knfrc" event={"ID":"468850b2-f30d-49d0-aa49-7be726bc68d7","Type":"ContainerStarted","Data":"916f03eb2a2d63edf53250c9cad53aa779d4cc646aadcfbbd0e689f94fd09b78"}
Sep 14 07:50:06.360432 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.360362    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-knfrc" event={"ID":"468850b2-f30d-49d0-aa49-7be726bc68d7","Type":"ContainerStarted","Data":"c19aded7ef38b4564e1858dad178edd327524026fa6e924af5ee1bdf3bb6a28e"}
Sep 14 07:50:06.364462 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.364441    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:50:06.417838 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.417382    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:50:06.466433 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.466334    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-868688f697-qjkcx" podUID="a6e9078b-48f9-490e-b0e8-4b9419554367" containerName="registry" containerID="cri-o://bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8" gracePeriod=30
Sep 14 07:50:06.752271 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.752246    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:50:06.841153 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841120    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841296 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841162    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841296 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841195    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841296 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841249    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841296 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841284    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841315    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841353    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841396    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kxdk7\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7\") pod \"a6e9078b-48f9-490e-b0e8-4b9419554367\" (UID: \"a6e9078b-48f9-490e-b0e8-4b9419554367\") "
Sep 14 07:50:06.841767 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841716    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:06.841880 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.841818    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:06.843719 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.843670    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:06.843836 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.843803    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:06.843896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.843873    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:06.844075 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.844056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:06.844156 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.844138    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7" (OuterVolumeSpecName: "kube-api-access-kxdk7") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "kube-api-access-kxdk7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:06.849680 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.849662    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a6e9078b-48f9-490e-b0e8-4b9419554367" (UID: "a6e9078b-48f9-490e-b0e8-4b9419554367"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:50:06.942096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942070    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-installation-pull-secrets\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942093    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942102    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-bound-sa-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942111    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-trusted-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942121    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a6e9078b-48f9-490e-b0e8-4b9419554367-image-registry-private-configuration\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942130    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxdk7\" (UniqueName: \"kubernetes.io/projected/a6e9078b-48f9-490e-b0e8-4b9419554367-kube-api-access-kxdk7\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942140    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a6e9078b-48f9-490e-b0e8-4b9419554367-registry-certificates\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:06.942431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:06.942148    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a6e9078b-48f9-490e-b0e8-4b9419554367-ca-trust-extracted\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:07.365607 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.365568    2577 generic.go:356] "Generic (PLEG): container finished" podID="a6e9078b-48f9-490e-b0e8-4b9419554367" containerID="bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8" exitCode=0
Sep 14 07:50:07.365793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.365613    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-868688f697-qjkcx"
Sep 14 07:50:07.365793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.365656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868688f697-qjkcx" event={"ID":"a6e9078b-48f9-490e-b0e8-4b9419554367","Type":"ContainerDied","Data":"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"}
Sep 14 07:50:07.365793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.365708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-868688f697-qjkcx" event={"ID":"a6e9078b-48f9-490e-b0e8-4b9419554367","Type":"ContainerDied","Data":"5fcf961aaa3c675f12ff5e19431d6c6d107bc2de7196a27e64d96660fe1ee93a"}
Sep 14 07:50:07.365793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.365729    2577 scope.go:117] "RemoveContainer" containerID="bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"
Sep 14 07:50:07.384413 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.384391    2577 scope.go:117] "RemoveContainer" containerID="bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"
Sep 14 07:50:07.384745 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:50:07.384718    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8\": container with ID starting with bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8 not found: ID does not exist" containerID="bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"
Sep 14 07:50:07.384800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.384756    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8"} err="failed to get container status \"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8\": rpc error: code = NotFound desc = could not find container \"bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8\": container with ID starting with bd8cf5032932852f231d43177c333a71b156afdeb5603a631a9a2eb952c9b7a8 not found: ID does not exist"
Sep 14 07:50:07.388458 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.388418    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-knfrc" podStartSLOduration=130.364527553 podStartE2EDuration="2m11.388405436s" totalImagesPullingTime="1.023877883s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:47:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:50:07.387892993 +0000 UTC m=+132.162391331" watchObservedRunningTime="2026-09-14 07:50:07.388405436 +0000 UTC m=+132.162903770"
Sep 14 07:50:07.403362 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.403340    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:50:07.406936 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.406906    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-868688f697-qjkcx"]
Sep 14 07:50:07.758974 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:07.758948    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6e9078b-48f9-490e-b0e8-4b9419554367" path="/var/lib/kubelet/pods/a6e9078b-48f9-490e-b0e8-4b9419554367/volumes"
Sep 14 07:50:13.384066 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.384005    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6c577b694f-n92h8" podUID="d7e9de41-1627-4cb6-b66b-fc09e5255352" containerName="console" containerID="cri-o://7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c" gracePeriod=15
Sep 14 07:50:13.649905 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.649850    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:50:13.797269 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797241    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kssxl\" (UniqueName: \"kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797289    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797347    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797362    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797387    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert\") pod \"d7e9de41-1627-4cb6-b66b-fc09e5255352\" (UID: \"d7e9de41-1627-4cb6-b66b-fc09e5255352\") "
Sep 14 07:50:13.797817 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config" (OuterVolumeSpecName: "console-config") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:13.797817 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797783    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca" (OuterVolumeSpecName: "service-ca") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:13.797995 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.797790    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:13.799540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.799514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl" (OuterVolumeSpecName: "kube-api-access-kssxl") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "kube-api-access-kssxl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:13.799540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.799523    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:13.799659 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.799581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d7e9de41-1627-4cb6-b66b-fc09e5255352" (UID: "d7e9de41-1627-4cb6-b66b-fc09e5255352"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:13.898170 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898133    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kssxl\" (UniqueName: \"kubernetes.io/projected/d7e9de41-1627-4cb6-b66b-fc09e5255352-kube-api-access-kssxl\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:13.898170 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898172    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:13.898354 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898182    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:13.898354 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898192    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:13.898354 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898201    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:13.898354 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:13.898208    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d7e9de41-1627-4cb6-b66b-fc09e5255352-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:14.392586 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.392552    2577 generic.go:356] "Generic (PLEG): container finished" podID="d7e9de41-1627-4cb6-b66b-fc09e5255352" containerID="7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c" exitCode=0
Sep 14 07:50:14.393004 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.392600    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c577b694f-n92h8"
Sep 14 07:50:14.393004 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.392607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c577b694f-n92h8" event={"ID":"d7e9de41-1627-4cb6-b66b-fc09e5255352","Type":"ContainerDied","Data":"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"}
Sep 14 07:50:14.393004 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.392646    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c577b694f-n92h8" event={"ID":"d7e9de41-1627-4cb6-b66b-fc09e5255352","Type":"ContainerDied","Data":"2dd58ae75915e7c36f66552a767c9eb6c8cf71a212e5252298d8025392a13861"}
Sep 14 07:50:14.393004 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.392664    2577 scope.go:117] "RemoveContainer" containerID="7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"
Sep 14 07:50:14.400851 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.400835    2577 scope.go:117] "RemoveContainer" containerID="7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"
Sep 14 07:50:14.401139 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:50:14.401119    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c\": container with ID starting with 7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c not found: ID does not exist" containerID="7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"
Sep 14 07:50:14.401208 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.401149    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c"} err="failed to get container status \"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c\": rpc error: code = NotFound desc = could not find container \"7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c\": container with ID starting with 7b5948a6833c504027006948928b0cced9f5793c6d54d7cac2d2f9aa8571ef9c not found: ID does not exist"
Sep 14 07:50:14.414182 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.414152    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:50:14.418989 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:14.418969    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6c577b694f-n92h8"]
Sep 14 07:50:15.758744 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:15.758716    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e9de41-1627-4cb6-b66b-fc09e5255352" path="/var/lib/kubelet/pods/d7e9de41-1627-4cb6-b66b-fc09e5255352/volumes"
Sep 14 07:50:19.411811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:19.411777    2577 generic.go:356] "Generic (PLEG): container finished" podID="15061a53-ef12-442c-a185-6f02c5fb5173" containerID="de3f7998fd1f3cf3ce9ddd56835a93bb673d49b88a7d93d111e4ebb8ff9258ce" exitCode=0
Sep 14 07:50:19.412235 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:19.411854    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d" event={"ID":"15061a53-ef12-442c-a185-6f02c5fb5173","Type":"ContainerDied","Data":"de3f7998fd1f3cf3ce9ddd56835a93bb673d49b88a7d93d111e4ebb8ff9258ce"}
Sep 14 07:50:19.412277 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:19.412234    2577 scope.go:117] "RemoveContainer" containerID="de3f7998fd1f3cf3ce9ddd56835a93bb673d49b88a7d93d111e4ebb8ff9258ce"
Sep 14 07:50:20.416775 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:20.416745    2577 generic.go:356] "Generic (PLEG): container finished" podID="23231f53-30d5-40c0-b1d8-41c236fcb9c6" containerID="125762bea5b596ae62c062ed9086c72c2f57e7a618373f42f00d5174012450c1" exitCode=0
Sep 14 07:50:20.417258 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:20.416820    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-zp4c6" event={"ID":"23231f53-30d5-40c0-b1d8-41c236fcb9c6","Type":"ContainerDied","Data":"125762bea5b596ae62c062ed9086c72c2f57e7a618373f42f00d5174012450c1"}
Sep 14 07:50:20.417258 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:20.417224    2577 scope.go:117] "RemoveContainer" containerID="125762bea5b596ae62c062ed9086c72c2f57e7a618373f42f00d5174012450c1"
Sep 14 07:50:20.418806 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:20.418779    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-cf7c776b7-t9x7d" event={"ID":"15061a53-ef12-442c-a185-6f02c5fb5173","Type":"ContainerStarted","Data":"5d366f34d0299a2a4bd49d5bb339a41d6175a3bd50a3fee3f23e329989353f9f"}
Sep 14 07:50:21.424341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:21.424302    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5844b856db-zp4c6" event={"ID":"23231f53-30d5-40c0-b1d8-41c236fcb9c6","Type":"ContainerStarted","Data":"b0c1aed1a6726d759bf40d501e2a17f1d921d1341ac59bdf767df1c973041c2e"}
Sep 14 07:50:31.438193 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.438134    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-584c7bf566-p8rhj" podUID="2dfe099f-df73-4cdc-8454-e93bd1890451" containerName="console" containerID="cri-o://ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee" gracePeriod=15
Sep 14 07:50:31.716448 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.716418    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:50:31.723393 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723363    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfg8x\" (UniqueName: \"kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723433    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723459    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723540 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723506    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723754 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723722    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723856 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723779    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723856 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723836    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config\") pod \"2dfe099f-df73-4cdc-8454-e93bd1890451\" (UID: \"2dfe099f-df73-4cdc-8454-e93bd1890451\") "
Sep 14 07:50:31.723994 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.723854    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:31.724085 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.724057    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.724177 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.724132    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca" (OuterVolumeSpecName: "service-ca") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:31.724242 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.724196    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config" (OuterVolumeSpecName: "console-config") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:31.724613 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.724584    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:50:31.725833 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.725800    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x" (OuterVolumeSpecName: "kube-api-access-dfg8x") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "kube-api-access-dfg8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:50:31.725931 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.725818    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:31.726075 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.726055    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "2dfe099f-df73-4cdc-8454-e93bd1890451" (UID: "2dfe099f-df73-4cdc-8454-e93bd1890451"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:50:31.824909 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824877    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.824909 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824906    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.824909 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824932    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.825222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824946    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/2dfe099f-df73-4cdc-8454-e93bd1890451-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.825222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824960    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfg8x\" (UniqueName: \"kubernetes.io/projected/2dfe099f-df73-4cdc-8454-e93bd1890451-kube-api-access-dfg8x\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:31.825222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:31.824974    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/2dfe099f-df73-4cdc-8454-e93bd1890451-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:50:32.468671 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.468633    2577 generic.go:356] "Generic (PLEG): container finished" podID="2dfe099f-df73-4cdc-8454-e93bd1890451" containerID="ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee" exitCode=0
Sep 14 07:50:32.469077 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.468729    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-584c7bf566-p8rhj"
Sep 14 07:50:32.469077 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.468762    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584c7bf566-p8rhj" event={"ID":"2dfe099f-df73-4cdc-8454-e93bd1890451","Type":"ContainerDied","Data":"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"}
Sep 14 07:50:32.469077 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.468810    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-584c7bf566-p8rhj" event={"ID":"2dfe099f-df73-4cdc-8454-e93bd1890451","Type":"ContainerDied","Data":"3a515eedbf3488fb038b6d19cdc36c9d8e55ed1105925ca216dc532f992d0a06"}
Sep 14 07:50:32.469077 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.468831    2577 scope.go:117] "RemoveContainer" containerID="ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"
Sep 14 07:50:32.477363 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.477341    2577 scope.go:117] "RemoveContainer" containerID="ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"
Sep 14 07:50:32.477661 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:50:32.477639    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee\": container with ID starting with ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee not found: ID does not exist" containerID="ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"
Sep 14 07:50:32.477718 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.477668    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee"} err="failed to get container status \"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee\": rpc error: code = NotFound desc = could not find container \"ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee\": container with ID starting with ef887b2e15343785d0962e02d6d772f12d1a3b205c3106b0376f37683c81a6ee not found: ID does not exist"
Sep 14 07:50:32.487241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.487217    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:50:32.491523 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:32.491502    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-584c7bf566-p8rhj"]
Sep 14 07:50:33.759242 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:50:33.759203    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2dfe099f-df73-4cdc-8454-e93bd1890451" path="/var/lib/kubelet/pods/2dfe099f-df73-4cdc-8454-e93bd1890451/volumes"
Sep 14 07:51:07.016103 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016068    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016352    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e9de41-1627-4cb6-b66b-fc09e5255352" containerName="console"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016364    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e9de41-1627-4cb6-b66b-fc09e5255352" containerName="console"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016380    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6e9078b-48f9-490e-b0e8-4b9419554367" containerName="registry"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016385    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6e9078b-48f9-490e-b0e8-4b9419554367" containerName="registry"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016393    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2dfe099f-df73-4cdc-8454-e93bd1890451" containerName="console"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016401    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2dfe099f-df73-4cdc-8454-e93bd1890451" containerName="console"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016458    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e9de41-1627-4cb6-b66b-fc09e5255352" containerName="console"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016465    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6e9078b-48f9-490e-b0e8-4b9419554367" containerName="registry"
Sep 14 07:51:07.016527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.016472    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="2dfe099f-df73-4cdc-8454-e93bd1890451" containerName="console"
Sep 14 07:51:07.021142 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.021121    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.025955 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.025908    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 14 07:51:07.026090 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026014    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 14 07:51:07.026244 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026225    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 14 07:51:07.026355 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026265    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 14 07:51:07.026355 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026240    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 14 07:51:07.026355 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026300    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 14 07:51:07.026596 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.026578    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 14 07:51:07.027533 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.027229    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 14 07:51:07.032341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.032322    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-59tdq\""
Sep 14 07:51:07.033078 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.033061    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 14 07:51:07.047245 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.047222    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:07.073203 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073183    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073208    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073242    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073302    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073334    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-out\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zcdw4\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-kube-api-access-zcdw4\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073570    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073608    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-web-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.073685 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.073646    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174305 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174277    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zcdw4\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-kube-api-access-zcdw4\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174314    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174343    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174372    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-web-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174403    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174434    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174472 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174455    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174475    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174553    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-out\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.174800 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174609    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.175135 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.174997    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.175687 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.175660    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.176463 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.176438    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/161ffc42-ea8b-4a77-abd4-eea3564164f3-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.177665 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.177455    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-web-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.177665 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.177566    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-volume\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.177795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.177681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.177839 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.177819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.178250 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.178223    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.178449 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.178429    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/161ffc42-ea8b-4a77-abd4-eea3564164f3-config-out\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.178950 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.178904    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.179720 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.179695    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-tls-assets\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.179720 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.179714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/161ffc42-ea8b-4a77-abd4-eea3564164f3-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.184059 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.184041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zcdw4\" (UniqueName: \"kubernetes.io/projected/161ffc42-ea8b-4a77-abd4-eea3564164f3-kube-api-access-zcdw4\") pod \"alertmanager-main-0\" (UID: \"161ffc42-ea8b-4a77-abd4-eea3564164f3\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.330879 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.330814    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 14 07:51:07.616569 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:07.616541    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 14 07:51:07.617530 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:51:07.617501    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod161ffc42_ea8b_4a77_abd4_eea3564164f3.slice/crio-213f67226aebcb58cdb2685900446e40388a8c9f4a178235361a708077a4bfaa WatchSource:0}: Error finding container 213f67226aebcb58cdb2685900446e40388a8c9f4a178235361a708077a4bfaa: Status 404 returned error can't find the container with id 213f67226aebcb58cdb2685900446e40388a8c9f4a178235361a708077a4bfaa
Sep 14 07:51:08.606452 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:08.606416    2577 generic.go:356] "Generic (PLEG): container finished" podID="161ffc42-ea8b-4a77-abd4-eea3564164f3" containerID="a6f16c6b8a9377bfc76783032c1fb2c3cd358bf75e21649941f4b329f930a72b" exitCode=0
Sep 14 07:51:08.606812 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:08.606504    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerDied","Data":"a6f16c6b8a9377bfc76783032c1fb2c3cd358bf75e21649941f4b329f930a72b"}
Sep 14 07:51:08.606812 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:08.606561    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"213f67226aebcb58cdb2685900446e40388a8c9f4a178235361a708077a4bfaa"}
Sep 14 07:51:10.619369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:10.619317    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"e3651a4d24a0c07600c6517ae730df90871bcfb549a664a98c12d3ba27f3b0c1"}
Sep 14 07:51:10.619674 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:10.619379    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"c83187ed333d118d9b121f76ca17001cef36a991a7963fe0da7e013d514287cb"}
Sep 14 07:51:10.619674 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:10.619411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"7e739ef6517788e10eebbdf97e64abdeb74010fed0253d8db8af3e85eb884d6b"}
Sep 14 07:51:10.619674 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:10.619438    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"c4c4fef7bf8ef46a0d78a0e5784d160146af80fe4dc85767f1bf024cd99838ba"}
Sep 14 07:51:11.629374 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:11.629338    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"ca84b98434f84dfd21df4a960be9a0c1791d4ff9852c80a256b46c4450f05647"}
Sep 14 07:51:11.629711 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:11.629406    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"161ffc42-ea8b-4a77-abd4-eea3564164f3","Type":"ContainerStarted","Data":"0bf58d99816dd737e18d06c59daa6d37b01902cb20a5959dcecbd9bb9ff374fc"}
Sep 14 07:51:11.663460 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:11.663379    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.135087192 podStartE2EDuration="5.663363846s" totalImagesPullingTime="2.513811368s" totalInitContainerRuntime="14.465286ms" isStatefulPod=false podCreationTimestamp="2026-09-14 07:51:06 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:51:11.662775023 +0000 UTC m=+196.437273357" watchObservedRunningTime="2026-09-14 07:51:11.663363846 +0000 UTC m=+196.437862221"
Sep 14 07:51:12.294792 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.294762    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:51:12.297856 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.297841    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.312484 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.312462    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:51:12.420242 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420215    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420260    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420278    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420391 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7pjh\" (UniqueName: \"kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420543 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420417    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.420543 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.420438    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.521688 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.521654    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7pjh\" (UniqueName: \"kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.521809 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.521710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.521809 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.521729    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.521961 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.521936    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522040    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522119 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522071    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522119 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522098    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522465    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522617 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522561    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522749 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522724    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.522962 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.522939    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.524521 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.524500    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.524521 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.524505    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.531208 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.531185    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7pjh\" (UniqueName: \"kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh\") pod \"console-74cf857455-rhr9v\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") " pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.610809 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.610734    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:12.881502 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:12.881471    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:51:12.883691 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:51:12.883666    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79119387_9698_43e2_b033_488f0660f7d7.slice/crio-44991431660d59b5fdbbcbaa624f74d18b4ff04866afea5ab10f5f964c45c6ae WatchSource:0}: Error finding container 44991431660d59b5fdbbcbaa624f74d18b4ff04866afea5ab10f5f964c45c6ae: Status 404 returned error can't find the container with id 44991431660d59b5fdbbcbaa624f74d18b4ff04866afea5ab10f5f964c45c6ae
Sep 14 07:51:13.639308 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:13.639273    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf857455-rhr9v" event={"ID":"79119387-9698-43e2-b033-488f0660f7d7","Type":"ContainerStarted","Data":"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"}
Sep 14 07:51:13.639463 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:13.639336    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf857455-rhr9v" event={"ID":"79119387-9698-43e2-b033-488f0660f7d7","Type":"ContainerStarted","Data":"44991431660d59b5fdbbcbaa624f74d18b4ff04866afea5ab10f5f964c45c6ae"}
Sep 14 07:51:13.661931 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:13.661872    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74cf857455-rhr9v" podStartSLOduration=1.6618563819999999 podStartE2EDuration="1.661856382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:51:13.660544146 +0000 UTC m=+198.435042492" watchObservedRunningTime="2026-09-14 07:51:13.661856382 +0000 UTC m=+198.436354716"
Sep 14 07:51:22.611285 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:22.611247    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:22.611648 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:22.611350    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:22.615942 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:22.615906    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:22.675028 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:22.674999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:51:22.730143 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:22.730109    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:51:47.749007 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:47.748948    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b545c7558-drgw7" podUID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" containerName="console" containerID="cri-o://a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0" gracePeriod=15
Sep 14 07:51:47.995346 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:47.995322    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:51:48.105896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.105896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105846    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2t5qm\" (UniqueName: \"kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.105896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105883    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.106201 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105933    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.106201 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105952    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.106201 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.105983    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.106201 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.106035    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config\") pod \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\" (UID: \"4a65ef9d-c056-48ee-85bf-6debed7dc45d\") "
Sep 14 07:51:48.106407 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.106294    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:48.106542 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.106500    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:48.106542 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.106514    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config" (OuterVolumeSpecName: "console-config") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:48.106542 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.106507    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:51:48.108161 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.108140    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:48.108561 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.108542    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:51:48.108652 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.108573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm" (OuterVolumeSpecName: "kube-api-access-2t5qm") pod "4a65ef9d-c056-48ee-85bf-6debed7dc45d" (UID: "4a65ef9d-c056-48ee-85bf-6debed7dc45d"). InnerVolumeSpecName "kube-api-access-2t5qm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:51:48.207059 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207034    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207059 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207057    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207218 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207066    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2t5qm\" (UniqueName: \"kubernetes.io/projected/4a65ef9d-c056-48ee-85bf-6debed7dc45d-kube-api-access-2t5qm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207218 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207075    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207218 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207084    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207218 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207092    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a65ef9d-c056-48ee-85bf-6debed7dc45d-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.207218 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.207100    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a65ef9d-c056-48ee-85bf-6debed7dc45d-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:51:48.771967 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.771908    2577 generic.go:356] "Generic (PLEG): container finished" podID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" containerID="a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0" exitCode=0
Sep 14 07:51:48.772341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.771977    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b545c7558-drgw7"
Sep 14 07:51:48.772341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.771987    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b545c7558-drgw7" event={"ID":"4a65ef9d-c056-48ee-85bf-6debed7dc45d","Type":"ContainerDied","Data":"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"}
Sep 14 07:51:48.772341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.772032    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b545c7558-drgw7" event={"ID":"4a65ef9d-c056-48ee-85bf-6debed7dc45d","Type":"ContainerDied","Data":"c86392bbc4f4ddf052001ddf460fe76d0aeb947d0d92ff3a9cf4d00e52dc496a"}
Sep 14 07:51:48.772341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.772058    2577 scope.go:117] "RemoveContainer" containerID="a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"
Sep 14 07:51:48.780445 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.780422    2577 scope.go:117] "RemoveContainer" containerID="a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"
Sep 14 07:51:48.780709 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:51:48.780688    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0\": container with ID starting with a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0 not found: ID does not exist" containerID="a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"
Sep 14 07:51:48.780760 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.780719    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0"} err="failed to get container status \"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0\": rpc error: code = NotFound desc = could not find container \"a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0\": container with ID starting with a55d490a6975f12f378aed22849f1b2f7c405daaf3a991e12723ea6be8200ed0 not found: ID does not exist"
Sep 14 07:51:48.794337 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.794315    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:51:48.798173 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:48.798154    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b545c7558-drgw7"]
Sep 14 07:51:49.758600 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:51:49.758564    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" path="/var/lib/kubelet/pods/4a65ef9d-c056-48ee-85bf-6debed7dc45d/volumes"
Sep 14 07:52:38.560330 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.560300    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"]
Sep 14 07:52:38.560736 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.560596    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" containerName="console"
Sep 14 07:52:38.560736 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.560609    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" containerName="console"
Sep 14 07:52:38.560736 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.560661    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a65ef9d-c056-48ee-85bf-6debed7dc45d" containerName="console"
Sep 14 07:52:38.563300 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.563283    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.566056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.566036    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 14 07:52:38.567084 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.567060    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-xzt54\""
Sep 14 07:52:38.567181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.567068    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 14 07:52:38.573408 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.573389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"]
Sep 14 07:52:38.670387 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.670352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrs5\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-kube-api-access-vwrs5\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.670614 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.670395    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.771051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.771016    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.771190 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.771092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrs5\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-kube-api-access-vwrs5\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.779873 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.779850    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.780190 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.780170    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrs5\" (UniqueName: \"kubernetes.io/projected/f1c4b235-d4c2-4747-99be-1a5b1fba2376-kube-api-access-vwrs5\") pod \"cert-manager-cainjector-559cdd8c87-h7gpb\" (UID: \"f1c4b235-d4c2-4747-99be-1a5b1fba2376\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:38.871807 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:38.871722    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"
Sep 14 07:52:39.142075 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:39.142000    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb"]
Sep 14 07:52:39.145143 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:52:39.145113    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf1c4b235_d4c2_4747_99be_1a5b1fba2376.slice/crio-21c9b59ea84ec98067a22f05cd44261a7d1b2996a3d14107281eb0bc1b93fff3 WatchSource:0}: Error finding container 21c9b59ea84ec98067a22f05cd44261a7d1b2996a3d14107281eb0bc1b93fff3: Status 404 returned error can't find the container with id 21c9b59ea84ec98067a22f05cd44261a7d1b2996a3d14107281eb0bc1b93fff3
Sep 14 07:52:39.964245 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:39.964201    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb" event={"ID":"f1c4b235-d4c2-4747-99be-1a5b1fba2376","Type":"ContainerStarted","Data":"21c9b59ea84ec98067a22f05cd44261a7d1b2996a3d14107281eb0bc1b93fff3"}
Sep 14 07:52:42.979497 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:42.979462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb" event={"ID":"f1c4b235-d4c2-4747-99be-1a5b1fba2376","Type":"ContainerStarted","Data":"c4f05392379b2f76cd552c2cd941e003d95de1201729adf7c6d266872c5dc916"}
Sep 14 07:52:43.001510 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:43.001459    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-h7gpb" podStartSLOduration=1.830806032 podStartE2EDuration="5.00144422s" totalImagesPullingTime="3.170638188s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:43.000223384 +0000 UTC m=+287.774721752" watchObservedRunningTime="2026-09-14 07:52:43.00144422 +0000 UTC m=+287.775942551"
Sep 14 07:52:43.900477 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:43.900444    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6kcmm"]
Sep 14 07:52:43.903686 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:43.903670    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:43.906409 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:43.906387    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-bv4l7\""
Sep 14 07:52:43.911832 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:43.911812    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6kcmm"]
Sep 14 07:52:44.013096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.013066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.013096 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.013100    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j5qnb\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-kube-api-access-j5qnb\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.113687 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.113655    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.113849 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.113693    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j5qnb\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-kube-api-access-j5qnb\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.123098 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.123068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.123208 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.123116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j5qnb\" (UniqueName: \"kubernetes.io/projected/668f8310-3605-4505-bb4b-3ec92c147c2e-kube-api-access-j5qnb\") pod \"cert-manager-webhook-6486ffd997-6kcmm\" (UID: \"668f8310-3605-4505-bb4b-3ec92c147c2e\") " pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.212336 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.212276    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:44.480505 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.480428    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-6kcmm"]
Sep 14 07:52:44.484307 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:52:44.484284    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod668f8310_3605_4505_bb4b_3ec92c147c2e.slice/crio-e9976a64cdc4ba72c8b6b2d880754a4f080b7e3983ec924c085e1a30ae16df60 WatchSource:0}: Error finding container e9976a64cdc4ba72c8b6b2d880754a4f080b7e3983ec924c085e1a30ae16df60: Status 404 returned error can't find the container with id e9976a64cdc4ba72c8b6b2d880754a4f080b7e3983ec924c085e1a30ae16df60
Sep 14 07:52:44.988715 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.988677    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm" event={"ID":"668f8310-3605-4505-bb4b-3ec92c147c2e","Type":"ContainerStarted","Data":"de15a30914620180a9386777d220eff054464f5e9382c1aa8dbcedeb76358b71"}
Sep 14 07:52:44.988882 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.988724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm" event={"ID":"668f8310-3605-4505-bb4b-3ec92c147c2e","Type":"ContainerStarted","Data":"e9976a64cdc4ba72c8b6b2d880754a4f080b7e3983ec924c085e1a30ae16df60"}
Sep 14 07:52:44.988882 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:44.988748    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:46.888576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:46.888524    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm" podStartSLOduration=3.888506334 podStartE2EDuration="3.888506334s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:45.027179565 +0000 UTC m=+289.801677900" watchObservedRunningTime="2026-09-14 07:52:46.888506334 +0000 UTC m=+291.663004669"
Sep 14 07:52:46.888966 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:46.888641    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbq2r"]
Sep 14 07:52:46.891984 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:46.891967    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:46.895581 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:46.895557    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9p7rr\""
Sep 14 07:52:46.906860 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:46.906835    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbq2r"]
Sep 14 07:52:47.037759 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.037724    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.037759 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.037759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c46l8\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-kube-api-access-c46l8\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.139022 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.138939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.139022 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.138977    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c46l8\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-kube-api-access-c46l8\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.154576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.154547    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-bound-sa-token\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.154930 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.154899    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c46l8\" (UniqueName: \"kubernetes.io/projected/2230620b-a251-43f3-ba3c-c04f4077167f-kube-api-access-c46l8\") pod \"cert-manager-55b4d954d5-xbq2r\" (UID: \"2230620b-a251-43f3-ba3c-c04f4077167f\") " pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.200463 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.200433    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-xbq2r"
Sep 14 07:52:47.470116 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:47.470094    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-xbq2r"]
Sep 14 07:52:47.472319 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:52:47.472296    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2230620b_a251_43f3_ba3c_c04f4077167f.slice/crio-40cda5d1627425bc9f8b229a951089035c3759902ff7381594cd638ceb2f6786 WatchSource:0}: Error finding container 40cda5d1627425bc9f8b229a951089035c3759902ff7381594cd638ceb2f6786: Status 404 returned error can't find the container with id 40cda5d1627425bc9f8b229a951089035c3759902ff7381594cd638ceb2f6786
Sep 14 07:52:48.001676 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:48.001643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbq2r" event={"ID":"2230620b-a251-43f3-ba3c-c04f4077167f","Type":"ContainerStarted","Data":"3ec154dc7f037a0eba2e19b3b151218182a4d33167900830fd96b3389352222f"}
Sep 14 07:52:48.002064 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:48.001686    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-xbq2r" event={"ID":"2230620b-a251-43f3-ba3c-c04f4077167f","Type":"ContainerStarted","Data":"40cda5d1627425bc9f8b229a951089035c3759902ff7381594cd638ceb2f6786"}
Sep 14 07:52:48.019551 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:48.019509    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-xbq2r" podStartSLOduration=2.01949409 podStartE2EDuration="2.01949409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:52:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:52:48.018726693 +0000 UTC m=+292.793225053" watchObservedRunningTime="2026-09-14 07:52:48.01949409 +0000 UTC m=+292.793992424"
Sep 14 07:52:50.995169 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:50.995139    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-6kcmm"
Sep 14 07:52:55.657655 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:55.657623    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:52:55.666389 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:55.666364    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:52:55.670131 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:52:55.670101    2577 kubelet.go:1648] "Image garbage collection succeeded"
Sep 14 07:53:13.211538 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.211501    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"]
Sep 14 07:53:13.215027 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.215011    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.217748 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.217725    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 14 07:53:13.217869 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.217794    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 14 07:53:13.217945 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.217889    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 14 07:53:13.217945 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.217939    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 14 07:53:13.218055 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.218032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-79xct\""
Sep 14 07:53:13.233661 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.233639    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"]
Sep 14 07:53:13.257867 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.257835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjq7\" (UniqueName: \"kubernetes.io/projected/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-kube-api-access-5wjq7\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.258013 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.257872    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.258013 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.257954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.258013 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.257973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.258139 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.258039    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.358820 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.358777    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.359040 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.358841    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.359040 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.358865    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjq7\" (UniqueName: \"kubernetes.io/projected/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-kube-api-access-5wjq7\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.359040 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.358885    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.359040 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.358948    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.359347 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.359329    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-manifests\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.361239 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.361209    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-tmp\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.361451 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.361436    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.361582 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.361559    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-webhook-cert\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.368682 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.368657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjq7\" (UniqueName: \"kubernetes.io/projected/70de97a6-a0b1-4859-b7cf-ee36d84c41b5-kube-api-access-5wjq7\") pod \"opendatahub-operator-controller-manager-84c4c5b687-6w2gs\" (UID: \"70de97a6-a0b1-4859-b7cf-ee36d84c41b5\") " pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.526022 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.525912    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:13.824558 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.824537    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"]
Sep 14 07:53:13.826229 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:53:13.826203    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70de97a6_a0b1_4859_b7cf_ee36d84c41b5.slice/crio-94d2b486313a2103944266a8cacdd184da746e4edb3e6db7cabf3205b54a3973 WatchSource:0}: Error finding container 94d2b486313a2103944266a8cacdd184da746e4edb3e6db7cabf3205b54a3973: Status 404 returned error can't find the container with id 94d2b486313a2103944266a8cacdd184da746e4edb3e6db7cabf3205b54a3973
Sep 14 07:53:13.828010 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:13.827990    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:53:14.104300 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:14.104214    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs" event={"ID":"70de97a6-a0b1-4859-b7cf-ee36d84c41b5","Type":"ContainerStarted","Data":"94d2b486313a2103944266a8cacdd184da746e4edb3e6db7cabf3205b54a3973"}
Sep 14 07:53:17.125542 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:17.125459    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs" event={"ID":"70de97a6-a0b1-4859-b7cf-ee36d84c41b5","Type":"ContainerStarted","Data":"dddcd25a3f87053e5366fbecd767557ecc82cd69c2ed2322c26e0027a3f28673"}
Sep 14 07:53:18.131560 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:18.131476    2577 generic.go:356] "Generic (PLEG): container finished" podID="70de97a6-a0b1-4859-b7cf-ee36d84c41b5" containerID="dddcd25a3f87053e5366fbecd767557ecc82cd69c2ed2322c26e0027a3f28673" exitCode=0
Sep 14 07:53:18.131560 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:18.131542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs" event={"ID":"70de97a6-a0b1-4859-b7cf-ee36d84c41b5","Type":"ContainerDied","Data":"dddcd25a3f87053e5366fbecd767557ecc82cd69c2ed2322c26e0027a3f28673"}
Sep 14 07:53:19.138597 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:19.138511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs" event={"ID":"70de97a6-a0b1-4859-b7cf-ee36d84c41b5","Type":"ContainerStarted","Data":"cb3af8c9bdab7d735713fb75c4770f79e6c04fb86ef4fe038595432a29f9da3b"}
Sep 14 07:53:19.139008 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:19.138647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:19.182927 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:19.182877    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs" podStartSLOduration=1.736496336 podStartE2EDuration="6.182860065s" totalImagesPullingTime="3.433168145s" totalInitContainerRuntime="1.013195584s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:13 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:19.181523105 +0000 UTC m=+323.956021441" watchObservedRunningTime="2026-09-14 07:53:19.182860065 +0000 UTC m=+323.957358451"
Sep 14 07:53:30.145127 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:30.145093    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-84c4c5b687-6w2gs"
Sep 14 07:53:32.854946 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.854898    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"]
Sep 14 07:53:32.863327 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.863300    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:32.868306 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.868280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 14 07:53:32.868439 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.868308    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 14 07:53:32.869495 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.869472    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 14 07:53:32.869614 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.869494    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 14 07:53:32.869614 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.869476    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-nplqb\""
Sep 14 07:53:32.879941 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.879906    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"]
Sep 14 07:53:32.923222 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.923200    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tmp\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:32.923330 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.923228    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nw2g\" (UniqueName: \"kubernetes.io/projected/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-kube-api-access-6nw2g\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:32.923330 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:32.923256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.024295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.024266    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tmp\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.024295 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.024299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nw2g\" (UniqueName: \"kubernetes.io/projected/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-kube-api-access-6nw2g\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.024521 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.024332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.026649 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.026627    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tmp\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.026906 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.026890    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-tls-certs\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.035828 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.035803    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nw2g\" (UniqueName: \"kubernetes.io/projected/57ec6ca2-c92d-4e64-a704-e94f36ef1d0f-kube-api-access-6nw2g\") pod \"kube-auth-proxy-86f4846dcd-ls8br\" (UID: \"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f\") " pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.173292 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.173261    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"
Sep 14 07:53:33.451580 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:33.451557    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br"]
Sep 14 07:53:33.453323 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:53:33.453293    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ec6ca2_c92d_4e64_a704_e94f36ef1d0f.slice/crio-08f6668965cade5e7f51c8b1e1377d168654a6919fbfa72cba9284d21818ed97 WatchSource:0}: Error finding container 08f6668965cade5e7f51c8b1e1377d168654a6919fbfa72cba9284d21818ed97: Status 404 returned error can't find the container with id 08f6668965cade5e7f51c8b1e1377d168654a6919fbfa72cba9284d21818ed97
Sep 14 07:53:34.206829 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:34.206788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br" event={"ID":"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f","Type":"ContainerStarted","Data":"08f6668965cade5e7f51c8b1e1377d168654a6919fbfa72cba9284d21818ed97"}
Sep 14 07:53:36.217420 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:36.217378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br" event={"ID":"57ec6ca2-c92d-4e64-a704-e94f36ef1d0f","Type":"ContainerStarted","Data":"b6ed661cf092813a79e21a1af58ac032cfc7e9acd50fbe550b4634de31d2c698"}
Sep 14 07:53:36.244853 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:36.244806    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-86f4846dcd-ls8br" podStartSLOduration=1.5698466629999999 podStartE2EDuration="4.24478969s" totalImagesPullingTime="2.674943027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:36.242448908 +0000 UTC m=+341.016947244" watchObservedRunningTime="2026-09-14 07:53:36.24478969 +0000 UTC m=+341.019288025"
Sep 14 07:53:39.260445 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.260415    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:39.268565 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.268536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.271422 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.271400    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 07:53:39.271577 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.271556    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-s4z8k\""
Sep 14 07:53:39.277082 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.277056    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:39.376622 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376592    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376777 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376626    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dxbk\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376777 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376652    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376777 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376759    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376819    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376963    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.376992 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.376984    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.477935 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.477883    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.477942    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.477981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478006    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dxbk\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478035    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478302 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478096    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478302 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478150    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478302 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478182    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478302 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478221    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478502 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478300    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478703 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478680    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478803 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478709    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.478891 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.478870    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.479241 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.479219    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.480431 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.480409    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.480648 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.480630    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.487346 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.487325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.487646 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.487624    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dxbk\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk\") pod \"data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.579695 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.579617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:39.858182 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:39.858158    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:39.860815 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:53:39.860788    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bf3a0f1_257a_4f64_95a4_5f02835eb7d4.slice/crio-a24cf8587d954d5f517a1a2d273ade476c0d5c93d69bb2492f8cbe35d4efd8a5 WatchSource:0}: Error finding container a24cf8587d954d5f517a1a2d273ade476c0d5c93d69bb2492f8cbe35d4efd8a5: Status 404 returned error can't find the container with id a24cf8587d954d5f517a1a2d273ade476c0d5c93d69bb2492f8cbe35d4efd8a5
Sep 14 07:53:40.236479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:40.236442    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" event={"ID":"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4","Type":"ContainerStarted","Data":"a24cf8587d954d5f517a1a2d273ade476c0d5c93d69bb2492f8cbe35d4efd8a5"}
Sep 14 07:53:42.606338 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:42.606298    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 07:53:42.606696 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:42.606394    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 07:53:42.606696 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:42.606444    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 07:53:43.233270 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.233240    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n9q5h"]
Sep 14 07:53:43.237004 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.236976    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.239795 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.239772    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 14 07:53:43.239957 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.239939    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-hrd68\""
Sep 14 07:53:43.248881 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.248857    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n9q5h"]
Sep 14 07:53:43.253261 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.253237    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" event={"ID":"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4","Type":"ContainerStarted","Data":"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"}
Sep 14 07:53:43.287147 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.287105    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" podStartSLOduration=1.54373986 podStartE2EDuration="4.287089541s" totalImagesPullingTime="2.743349681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:43.285964244 +0000 UTC m=+348.060462589" watchObservedRunningTime="2026-09-14 07:53:43.287089541 +0000 UTC m=+348.061587879"
Sep 14 07:53:43.309384 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.309352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde2699e-2559-4f06-a35f-56e4eabb8d73-cert\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.309513 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.309485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5w2t6\" (UniqueName: \"kubernetes.io/projected/fde2699e-2559-4f06-a35f-56e4eabb8d73-kube-api-access-5w2t6\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.410883 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.410849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde2699e-2559-4f06-a35f-56e4eabb8d73-cert\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.411069 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.410889    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5w2t6\" (UniqueName: \"kubernetes.io/projected/fde2699e-2559-4f06-a35f-56e4eabb8d73-kube-api-access-5w2t6\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.413546 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.413515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fde2699e-2559-4f06-a35f-56e4eabb8d73-cert\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.420142 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.420116    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5w2t6\" (UniqueName: \"kubernetes.io/projected/fde2699e-2559-4f06-a35f-56e4eabb8d73-kube-api-access-5w2t6\") pod \"odh-model-controller-7998c67dc4-n9q5h\" (UID: \"fde2699e-2559-4f06-a35f-56e4eabb8d73\") " pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.549591 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.549491    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:43.580211 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.580155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:43.581560 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.581530    2577 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 14 07:53:43.581655 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.581595    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 14 07:53:43.824941 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:43.824850    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-n9q5h"]
Sep 14 07:53:43.828021 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:53:43.827980    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfde2699e_2559_4f06_a35f_56e4eabb8d73.slice/crio-bbe9d5de44d166a33e09498d75a022f3fab455b1134e82e7a90dc0e1719f109c WatchSource:0}: Error finding container bbe9d5de44d166a33e09498d75a022f3fab455b1134e82e7a90dc0e1719f109c: Status 404 returned error can't find the container with id bbe9d5de44d166a33e09498d75a022f3fab455b1134e82e7a90dc0e1719f109c
Sep 14 07:53:44.260539 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:44.260503    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h" event={"ID":"fde2699e-2559-4f06-a35f-56e4eabb8d73","Type":"ContainerStarted","Data":"bbe9d5de44d166a33e09498d75a022f3fab455b1134e82e7a90dc0e1719f109c"}
Sep 14 07:53:44.580830 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:44.580527    2577 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 14 07:53:44.580830 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:44.580611    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 14 07:53:44.680964 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:44.680862    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:45.580425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:45.579999    2577 patch_prober.go:28] interesting pod/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused" start-of-body=
Sep 14 07:53:45.580425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:45.580076    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.30:15021/healthz/ready\": dial tcp 10.132.0.30:15021: connect: connection refused"
Sep 14 07:53:46.271997 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:46.271911    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy" containerID="cri-o://0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26" gracePeriod=30
Sep 14 07:53:47.278301 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:47.278264    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h" event={"ID":"fde2699e-2559-4f06-a35f-56e4eabb8d73","Type":"ContainerStarted","Data":"c5500754eceab9482c6df82ace82eaee2579a42b21d98aeffe33e61091c2e88a"}
Sep 14 07:53:47.278679 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:47.278391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:53:47.297329 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:47.297278    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h" podStartSLOduration=1.6727919 podStartE2EDuration="4.297259579s" totalImagesPullingTime="2.624467679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:47.296443358 +0000 UTC m=+352.070941695" watchObservedRunningTime="2026-09-14 07:53:47.297259579 +0000 UTC m=+352.071757913"
Sep 14 07:53:50.090581 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.090497    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"]
Sep 14 07:53:50.098928 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.098892    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.101836 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.101812    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 14 07:53:50.102159 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.102138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"]
Sep 14 07:53:50.103033 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.103012    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 14 07:53:50.103155 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.103026    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 14 07:53:50.103155 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.103036    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-fcpdl\""
Sep 14 07:53:50.103155 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.103121    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 14 07:53:50.103326 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.103038    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 14 07:53:50.175475 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.175445    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-metrics-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.175625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.175524    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pctm\" (UniqueName: \"kubernetes.io/projected/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-kube-api-access-9pctm\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.175625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.175549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-manager-config\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.175625 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.175573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.276551 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.276508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pctm\" (UniqueName: \"kubernetes.io/projected/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-kube-api-access-9pctm\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.276551 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.276552    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-manager-config\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.276766 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.276590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.276766 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.276643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-metrics-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.277297 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.277271    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-manager-config\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.279167 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.279146    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.279245 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.279173    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-metrics-cert\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.286741 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.286717    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pctm\" (UniqueName: \"kubernetes.io/projected/f91d4d29-0894-4215-b7c4-6cbd17b73bc5-kube-api-access-9pctm\") pod \"lws-controller-manager-784955c84b-bblsp\" (UID: \"f91d4d29-0894-4215-b7c4-6cbd17b73bc5\") " pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.408414 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.408387    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:50.736079 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:50.736052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"]
Sep 14 07:53:50.738093 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:53:50.738064    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf91d4d29_0894_4215_b7c4_6cbd17b73bc5.slice/crio-9d33e156e39e5ee6d49a1fc2ec501e87b788ef32f6927ae0c72041dbfa49f5b9 WatchSource:0}: Error finding container 9d33e156e39e5ee6d49a1fc2ec501e87b788ef32f6927ae0c72041dbfa49f5b9: Status 404 returned error can't find the container with id 9d33e156e39e5ee6d49a1fc2ec501e87b788ef32f6927ae0c72041dbfa49f5b9
Sep 14 07:53:51.299380 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.299352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp" event={"ID":"f91d4d29-0894-4215-b7c4-6cbd17b73bc5","Type":"ContainerStarted","Data":"9d33e156e39e5ee6d49a1fc2ec501e87b788ef32f6927ae0c72041dbfa49f5b9"}
Sep 14 07:53:51.540584 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.540509    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:51.689181 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689146    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689193    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689227    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689258    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689369 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689297    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689594 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689382    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dxbk\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689594 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689436    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689594 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689471    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689594 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689522    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token\") pod \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\" (UID: \"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4\") "
Sep 14 07:53:51.689841 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689743    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:51.689900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689851    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data" (OuterVolumeSpecName: "istio-data") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:51.689900 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.689837    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:53:51.690965 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.690795    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:51.690965 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.690815    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:51.692741 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.692243    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk" (OuterVolumeSpecName: "kube-api-access-9dxbk") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "kube-api-access-9dxbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:51.692741 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.692391    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.693192 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.692881    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token" (OuterVolumeSpecName: "istio-token") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:53:51.695449 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.695421    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 07:53:51.697298 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.697027    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" (UID: "0bf3a0f1-257a-4f64-95a4-5f02835eb7d4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.792945    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.792973    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.792982    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.792994    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.793006    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.793020    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dxbk\" (UniqueName: \"kubernetes.io/projected/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-kube-api-access-9dxbk\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.793037    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:51.793112 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:51.793052    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:53:52.307689 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.307652    2577 generic.go:356] "Generic (PLEG): container finished" podID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerID="0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26" exitCode=0
Sep 14 07:53:52.308117 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.307722    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"
Sep 14 07:53:52.308117 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.307737    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" event={"ID":"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4","Type":"ContainerDied","Data":"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"}
Sep 14 07:53:52.308117 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.307811    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9" event={"ID":"0bf3a0f1-257a-4f64-95a4-5f02835eb7d4","Type":"ContainerDied","Data":"a24cf8587d954d5f517a1a2d273ade476c0d5c93d69bb2492f8cbe35d4efd8a5"}
Sep 14 07:53:52.308117 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.307842    2577 scope.go:117] "RemoveContainer" containerID="0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"
Sep 14 07:53:52.332694 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.332668    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:52.336527 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:52.336503    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6745bb66bcr5mx9"]
Sep 14 07:53:53.053870 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.053692    2577 scope.go:117] "RemoveContainer" containerID="0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"
Sep 14 07:53:53.054112 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:53:53.054090    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26\": container with ID starting with 0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26 not found: ID does not exist" containerID="0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"
Sep 14 07:53:53.054165 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.054123    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26"} err="failed to get container status \"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26\": rpc error: code = NotFound desc = could not find container \"0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26\": container with ID starting with 0c691db02f1f5abd688d9da5e99edc42d764add68968433d1b2243621bcc4c26 not found: ID does not exist"
Sep 14 07:53:53.313987 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.313862    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp" event={"ID":"f91d4d29-0894-4215-b7c4-6cbd17b73bc5","Type":"ContainerStarted","Data":"8efb01f12e60abcfba1172649399576bc834ca3c0f18da7374f1af69b73d3b71"}
Sep 14 07:53:53.313987 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.313968    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:53:53.338988 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.338943    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp" podStartSLOduration=0.966284169 podStartE2EDuration="3.338901561s" totalImagesPullingTime="2.372617392s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:53:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:53:53.338763401 +0000 UTC m=+358.113261736" watchObservedRunningTime="2026-09-14 07:53:53.338901561 +0000 UTC m=+358.113399898"
Sep 14 07:53:53.763102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:53.763070    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" path="/var/lib/kubelet/pods/0bf3a0f1-257a-4f64-95a4-5f02835eb7d4/volumes"
Sep 14 07:53:58.285339 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:53:58.285304    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-n9q5h"
Sep 14 07:54:04.321711 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:54:04.321675    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-784955c84b-bblsp"
Sep 14 07:54:54.964829 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:54:54.964792    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:55:13.845479 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.845450    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"]
Sep 14 07:55:13.845941 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.845761    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy"
Sep 14 07:55:13.845941 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.845774    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy"
Sep 14 07:55:13.845941 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.845841    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bf3a0f1-257a-4f64-95a4-5f02835eb7d4" containerName="istio-proxy"
Sep 14 07:55:13.848783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.848766    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:13.851811 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.851789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-v4tmp\""
Sep 14 07:55:13.851896 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.851789    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 14 07:55:13.852206 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.852189    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 14 07:55:13.852261 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.852246    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 14 07:55:13.852311 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.852261    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 14 07:55:13.859520 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.859498    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"]
Sep 14 07:55:13.964574 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.964545    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:13.964738 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.964593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:13.964738 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:13.964641    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v77h9\" (UniqueName: \"kubernetes.io/projected/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-kube-api-access-v77h9\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.065455 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.065405    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.065636 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.065565    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.065708 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.065617    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v77h9\" (UniqueName: \"kubernetes.io/projected/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-kube-api-access-v77h9\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.065708 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:55:14.065679    2577 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 14 07:55:14.065793 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:55:14.065750    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert podName:bd7bf4c3-6882-4a2e-911f-0b08b5829b03 nodeName:}" failed. No retries permitted until 2026-09-14 07:55:14.565733143 +0000 UTC m=+439.340231455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert") pod "kuadrant-console-plugin-fbdd4f445-x87d6" (UID: "bd7bf4c3-6882-4a2e-911f-0b08b5829b03") : secret "plugin-serving-cert" not found
Sep 14 07:55:14.066071 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.066054    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.076984 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.076965    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v77h9\" (UniqueName: \"kubernetes.io/projected/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-kube-api-access-v77h9\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.568981 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.568949    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.571349 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.571320    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bd7bf4c3-6882-4a2e-911f-0b08b5829b03-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-x87d6\" (UID: \"bd7bf4c3-6882-4a2e-911f-0b08b5829b03\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:14.757266 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:14.757233    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"
Sep 14 07:55:15.034084 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:15.034052    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6"]
Sep 14 07:55:15.036862 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:55:15.036830    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbd7bf4c3_6882_4a2e_911f_0b08b5829b03.slice/crio-e34d55ab2a3a3781972e93a0d78f5984156056860df1ed0a3fdb343307443c31 WatchSource:0}: Error finding container e34d55ab2a3a3781972e93a0d78f5984156056860df1ed0a3fdb343307443c31: Status 404 returned error can't find the container with id e34d55ab2a3a3781972e93a0d78f5984156056860df1ed0a3fdb343307443c31
Sep 14 07:55:15.678730 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:15.678694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6" event={"ID":"bd7bf4c3-6882-4a2e-911f-0b08b5829b03","Type":"ContainerStarted","Data":"e34d55ab2a3a3781972e93a0d78f5984156056860df1ed0a3fdb343307443c31"}
Sep 14 07:55:19.989700 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:19.989601    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-74cf857455-rhr9v" podUID="79119387-9698-43e2-b033-488f0660f7d7" containerName="console" containerID="cri-o://f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f" gracePeriod=15
Sep 14 07:55:20.229520 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.229498    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:55:20.317217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317138    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317173    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7pjh\" (UniqueName: \"kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317214    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317239    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317263    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317305    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317349    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config\") pod \"79119387-9698-43e2-b033-488f0660f7d7\" (UID: \"79119387-9698-43e2-b033-488f0660f7d7\") "
Sep 14 07:55:20.317705 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317582    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:55:20.317769 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317748    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:55:20.317817 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.317768    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config" (OuterVolumeSpecName: "console-config") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:55:20.318036 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.318016    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca" (OuterVolumeSpecName: "service-ca") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 07:55:20.319397 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.319380    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:55:20.319778 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.319753    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 07:55:20.319778 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.319766    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh" (OuterVolumeSpecName: "kube-api-access-p7pjh") pod "79119387-9698-43e2-b033-488f0660f7d7" (UID: "79119387-9698-43e2-b033-488f0660f7d7"). InnerVolumeSpecName "kube-api-access-p7pjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:55:20.418351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418304    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-console-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418345    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418356    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-oauth-serving-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418365    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418601 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418374    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79119387-9698-43e2-b033-488f0660f7d7-console-oauth-config\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418601 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418382    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79119387-9698-43e2-b033-488f0660f7d7-trusted-ca-bundle\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.418601 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.418389    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7pjh\" (UniqueName: \"kubernetes.io/projected/79119387-9698-43e2-b033-488f0660f7d7-kube-api-access-p7pjh\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:55:20.704435 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.704406    2577 generic.go:356] "Generic (PLEG): container finished" podID="79119387-9698-43e2-b033-488f0660f7d7" containerID="f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f" exitCode=0
Sep 14 07:55:20.704567 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.704465    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74cf857455-rhr9v"
Sep 14 07:55:20.704567 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.704477    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf857455-rhr9v" event={"ID":"79119387-9698-43e2-b033-488f0660f7d7","Type":"ContainerDied","Data":"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"}
Sep 14 07:55:20.704567 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.704539    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74cf857455-rhr9v" event={"ID":"79119387-9698-43e2-b033-488f0660f7d7","Type":"ContainerDied","Data":"44991431660d59b5fdbbcbaa624f74d18b4ff04866afea5ab10f5f964c45c6ae"}
Sep 14 07:55:20.704567 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.704560    2577 scope.go:117] "RemoveContainer" containerID="f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"
Sep 14 07:55:20.706316 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.706284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6" event={"ID":"bd7bf4c3-6882-4a2e-911f-0b08b5829b03","Type":"ContainerStarted","Data":"26a148dea25a53861f6be16eca6bcbe20e9becb4412842af6cdca3153f2f3105"}
Sep 14 07:55:20.713582 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.713565    2577 scope.go:117] "RemoveContainer" containerID="f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"
Sep 14 07:55:20.713824 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:55:20.713805    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f\": container with ID starting with f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f not found: ID does not exist" containerID="f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"
Sep 14 07:55:20.713937 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.713828    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f"} err="failed to get container status \"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f\": rpc error: code = NotFound desc = could not find container \"f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f\": container with ID starting with f50a6b21bc1559faf4c7c2a894643db066761f9944b612dbe4f2a8cdb3900d0f not found: ID does not exist"
Sep 14 07:55:20.726475 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.726432    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-x87d6" podStartSLOduration=3.108041738 podStartE2EDuration="7.726419697s" totalImagesPullingTime="4.618377959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:55:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:55:20.725332396 +0000 UTC m=+445.499830727" watchObservedRunningTime="2026-09-14 07:55:20.726419697 +0000 UTC m=+445.500918032"
Sep 14 07:55:20.741963 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.741938    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:55:20.747644 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:20.747623    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74cf857455-rhr9v"]
Sep 14 07:55:21.760348 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:55:21.760319    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79119387-9698-43e2-b033-488f0660f7d7" path="/var/lib/kubelet/pods/79119387-9698-43e2-b033-488f0660f7d7/volumes"
Sep 14 07:56:08.315132 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.315104    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:08.315531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.315420    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79119387-9698-43e2-b033-488f0660f7d7" containerName="console"
Sep 14 07:56:08.315531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.315431    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="79119387-9698-43e2-b033-488f0660f7d7" containerName="console"
Sep 14 07:56:08.315531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.315499    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="79119387-9698-43e2-b033-488f0660f7d7" containerName="console"
Sep 14 07:56:08.336551 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.336523    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:08.336706 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.336610    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:08.339216 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.339195    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-ztf74\""
Sep 14 07:56:08.522582 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.522549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2z2x8\" (UniqueName: \"kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8\") pod \"authorino-7bcfd8f9cd-8pckn\" (UID: \"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:08.623263 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.623188    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2z2x8\" (UniqueName: \"kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8\") pod \"authorino-7bcfd8f9cd-8pckn\" (UID: \"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:08.633291 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.633265    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:08.636545 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.636530    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:08.638384 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.638364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2z2x8\" (UniqueName: \"kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8\") pod \"authorino-7bcfd8f9cd-8pckn\" (UID: \"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b\") " pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:08.644046 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.644026    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:08.646494 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.646476    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:08.825409 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.825376    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2thw\" (UniqueName: \"kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw\") pod \"authorino-59c974bd48-lmfnx\" (UID: \"5ea37f51-7963-4670-a58c-b933e2909a6d\") " pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:08.914949 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.914898    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:08.918574 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:56:08.918535    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10d6c758_fc0d_44b6_9e6c_e530d9f38a5b.slice/crio-0ac05874e57da991f8489b5be9cb1f8ab736fee43b8df891cf48036154813d18 WatchSource:0}: Error finding container 0ac05874e57da991f8489b5be9cb1f8ab736fee43b8df891cf48036154813d18: Status 404 returned error can't find the container with id 0ac05874e57da991f8489b5be9cb1f8ab736fee43b8df891cf48036154813d18
Sep 14 07:56:08.926602 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.926578    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2thw\" (UniqueName: \"kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw\") pod \"authorino-59c974bd48-lmfnx\" (UID: \"5ea37f51-7963-4670-a58c-b933e2909a6d\") " pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:08.935144 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.935119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2thw\" (UniqueName: \"kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw\") pod \"authorino-59c974bd48-lmfnx\" (UID: \"5ea37f51-7963-4670-a58c-b933e2909a6d\") " pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:08.954908 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:08.954883    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:09.222871 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:09.222783    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:09.225393 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:56:09.225365    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5ea37f51_7963_4670_a58c_b933e2909a6d.slice/crio-9b968a9d49963e28b57f6575f616860f2f4beccc93b6d6096527af501a1f57b2 WatchSource:0}: Error finding container 9b968a9d49963e28b57f6575f616860f2f4beccc93b6d6096527af501a1f57b2: Status 404 returned error can't find the container with id 9b968a9d49963e28b57f6575f616860f2f4beccc93b6d6096527af501a1f57b2
Sep 14 07:56:09.923006 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:09.922926    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lmfnx" event={"ID":"5ea37f51-7963-4670-a58c-b933e2909a6d","Type":"ContainerStarted","Data":"9b968a9d49963e28b57f6575f616860f2f4beccc93b6d6096527af501a1f57b2"}
Sep 14 07:56:09.925539 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:09.925511    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" event={"ID":"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b","Type":"ContainerStarted","Data":"0ac05874e57da991f8489b5be9cb1f8ab736fee43b8df891cf48036154813d18"}
Sep 14 07:56:12.942215 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:12.942181    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lmfnx" event={"ID":"5ea37f51-7963-4670-a58c-b933e2909a6d","Type":"ContainerStarted","Data":"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"}
Sep 14 07:56:12.943611 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:12.943591    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" event={"ID":"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b","Type":"ContainerStarted","Data":"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"}
Sep 14 07:56:12.958406 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:12.958367    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-lmfnx" podStartSLOduration=2.304200773 podStartE2EDuration="4.958354661s" totalImagesPullingTime="2.654153888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:12.956369006 +0000 UTC m=+497.730867341" watchObservedRunningTime="2026-09-14 07:56:12.958354661 +0000 UTC m=+497.732852996"
Sep 14 07:56:12.971489 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:12.971436    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" podStartSLOduration=2.02069213 podStartE2EDuration="4.971417494s" totalImagesPullingTime="2.950725364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:12.970018056 +0000 UTC m=+497.744516394" watchObservedRunningTime="2026-09-14 07:56:12.971417494 +0000 UTC m=+497.745915832"
Sep 14 07:56:13.000715 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:13.000681    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:14.953118 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:14.953071    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" podUID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" containerName="authorino" containerID="cri-o://a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2" gracePeriod=30
Sep 14 07:56:15.193368 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.193340    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:15.275069 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.274987    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2z2x8\" (UniqueName: \"kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8\") pod \"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b\" (UID: \"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b\") "
Sep 14 07:56:15.277170 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.277134    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8" (OuterVolumeSpecName: "kube-api-access-2z2x8") pod "10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" (UID: "10d6c758-fc0d-44b6-9e6c-e530d9f38a5b"). InnerVolumeSpecName "kube-api-access-2z2x8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:15.375698 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.375669    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2z2x8\" (UniqueName: \"kubernetes.io/projected/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b-kube-api-access-2z2x8\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:56:15.958720 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.958688    2577 generic.go:356] "Generic (PLEG): container finished" podID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" containerID="a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2" exitCode=0
Sep 14 07:56:15.959217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.958750    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn"
Sep 14 07:56:15.959217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.958778    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" event={"ID":"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b","Type":"ContainerDied","Data":"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"}
Sep 14 07:56:15.959217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.958814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-8pckn" event={"ID":"10d6c758-fc0d-44b6-9e6c-e530d9f38a5b","Type":"ContainerDied","Data":"0ac05874e57da991f8489b5be9cb1f8ab736fee43b8df891cf48036154813d18"}
Sep 14 07:56:15.959217 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.958830    2577 scope.go:117] "RemoveContainer" containerID="a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"
Sep 14 07:56:15.967545 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.967527    2577 scope.go:117] "RemoveContainer" containerID="a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"
Sep 14 07:56:15.967805 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:56:15.967783    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2\": container with ID starting with a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2 not found: ID does not exist" containerID="a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"
Sep 14 07:56:15.967859 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.967813    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2"} err="failed to get container status \"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2\": rpc error: code = NotFound desc = could not find container \"a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2\": container with ID starting with a54ad02a96d3caf9ccf72034e65ec5cd60dd188578ed5c0a8c466b35b7fa66c2 not found: ID does not exist"
Sep 14 07:56:15.977131 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.977109    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:15.984210 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:15.984188    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-8pckn"]
Sep 14 07:56:17.760144 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:17.760114    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" path="/var/lib/kubelet/pods/10d6c758-fc0d-44b6-9e6c-e530d9f38a5b/volumes"
Sep 14 07:56:36.018877 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.018837    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:36.019317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.019178    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" containerName="authorino"
Sep 14 07:56:36.019317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.019191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" containerName="authorino"
Sep 14 07:56:36.019317 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.019254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="10d6c758-fc0d-44b6-9e6c-e530d9f38a5b" containerName="authorino"
Sep 14 07:56:36.022192 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.022176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:36.029448 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.029287    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:36.144620 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.144587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjpw6\" (UniqueName: \"kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6\") pod \"authorino-557f876f69-f2vtz\" (UID: \"0cc9a802-97f4-42c9-8c67-da1f30113dbf\") " pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:36.245675 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.245643    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjpw6\" (UniqueName: \"kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6\") pod \"authorino-557f876f69-f2vtz\" (UID: \"0cc9a802-97f4-42c9-8c67-da1f30113dbf\") " pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:36.267362 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.267326    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjpw6\" (UniqueName: \"kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6\") pod \"authorino-557f876f69-f2vtz\" (UID: \"0cc9a802-97f4-42c9-8c67-da1f30113dbf\") " pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:36.294417 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.294344    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:36.294590 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.294578    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:36.332799 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.332763    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-55cc99f45f-fqxhh"]
Sep 14 07:56:36.336585 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.336563    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:36.348784 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.348727    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-55cc99f45f-fqxhh"]
Sep 14 07:56:36.399646 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.399615    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55cc99f45f-fqxhh"]
Sep 14 07:56:36.399888 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:56:36.399869    2577 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-49nj5], unattached volumes=[], failed to process volumes=[kube-api-access-49nj5]: context canceled" pod="kuadrant-system/authorino-55cc99f45f-fqxhh" podUID="876d7d7f-a79e-4453-b495-0960fb5d0e26"
Sep 14 07:56:36.444708 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.444677    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 07:56:36.447841 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.447823    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.449034 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.449012    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49nj5\" (UniqueName: \"kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5\") pod \"authorino-55cc99f45f-fqxhh\" (UID: \"876d7d7f-a79e-4453-b495-0960fb5d0e26\") " pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:36.464052 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.464032    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 14 07:56:36.478098 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.478068    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 07:56:36.550464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.550384    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49nj5\" (UniqueName: \"kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5\") pod \"authorino-55cc99f45f-fqxhh\" (UID: \"876d7d7f-a79e-4453-b495-0960fb5d0e26\") " pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:36.550464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.550431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.550464 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.550456    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm666\" (UniqueName: \"kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.560358 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.560328    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49nj5\" (UniqueName: \"kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5\") pod \"authorino-55cc99f45f-fqxhh\" (UID: \"876d7d7f-a79e-4453-b495-0960fb5d0e26\") " pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:36.565211 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.565188    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:36.568329 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:56:36.568297    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0cc9a802_97f4_42c9_8c67_da1f30113dbf.slice/crio-24037a346653bfa2fd178690a0d8351918e6be9567d7d5df5f5380378e42b60a WatchSource:0}: Error finding container 24037a346653bfa2fd178690a0d8351918e6be9567d7d5df5f5380378e42b60a: Status 404 returned error can't find the container with id 24037a346653bfa2fd178690a0d8351918e6be9567d7d5df5f5380378e42b60a
Sep 14 07:56:36.651531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.651501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.651531 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.651536    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm666\" (UniqueName: \"kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.653870 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.653843    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.660063 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.660041    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm666\" (UniqueName: \"kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666\") pod \"authorino-7568d7cd6f-78jbl\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") " pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:36.756099 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:36.756067    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 07:56:37.028712 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.028680    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 07:56:37.031110 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:56:37.031088    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b0fc740_bf99_4481_b1ca_6751cd586e42.slice/crio-57c94b081a29f158eb76e5e55833f15767f81ce29bf332cc1a3759865a66b086 WatchSource:0}: Error finding container 57c94b081a29f158eb76e5e55833f15767f81ce29bf332cc1a3759865a66b086: Status 404 returned error can't find the container with id 57c94b081a29f158eb76e5e55833f15767f81ce29bf332cc1a3759865a66b086
Sep 14 07:56:37.052461 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.052430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" event={"ID":"1b0fc740-bf99-4481-b1ca-6751cd586e42","Type":"ContainerStarted","Data":"57c94b081a29f158eb76e5e55833f15767f81ce29bf332cc1a3759865a66b086"}
Sep 14 07:56:37.054156 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.054136    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:37.054256 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.054157    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-f2vtz" podUID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" containerName="authorino" containerID="cri-o://57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4" gracePeriod=30
Sep 14 07:56:37.054308 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.054133    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-f2vtz" event={"ID":"0cc9a802-97f4-42c9-8c67-da1f30113dbf","Type":"ContainerStarted","Data":"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"}
Sep 14 07:56:37.054364 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.054316    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-f2vtz" event={"ID":"0cc9a802-97f4-42c9-8c67-da1f30113dbf","Type":"ContainerStarted","Data":"24037a346653bfa2fd178690a0d8351918e6be9567d7d5df5f5380378e42b60a"}
Sep 14 07:56:37.061612 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.061594    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:37.070727 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.070690    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-f2vtz" podStartSLOduration=0.65212508 podStartE2EDuration="1.070676363s" totalImagesPullingTime="418.551283ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:37.06925522 +0000 UTC m=+521.843753556" watchObservedRunningTime="2026-09-14 07:56:37.070676363 +0000 UTC m=+521.845174698"
Sep 14 07:56:37.155102 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.155073    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49nj5\" (UniqueName: \"kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5\") pod \"876d7d7f-a79e-4453-b495-0960fb5d0e26\" (UID: \"876d7d7f-a79e-4453-b495-0960fb5d0e26\") "
Sep 14 07:56:37.157090 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.157070    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5" (OuterVolumeSpecName: "kube-api-access-49nj5") pod "876d7d7f-a79e-4453-b495-0960fb5d0e26" (UID: "876d7d7f-a79e-4453-b495-0960fb5d0e26"). InnerVolumeSpecName "kube-api-access-49nj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:37.256255 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.256224    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49nj5\" (UniqueName: \"kubernetes.io/projected/876d7d7f-a79e-4453-b495-0960fb5d0e26-kube-api-access-49nj5\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:56:37.286484 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.286454    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:37.357128 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.357051    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjpw6\" (UniqueName: \"kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6\") pod \"0cc9a802-97f4-42c9-8c67-da1f30113dbf\" (UID: \"0cc9a802-97f4-42c9-8c67-da1f30113dbf\") "
Sep 14 07:56:37.359046 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.359024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6" (OuterVolumeSpecName: "kube-api-access-kjpw6") pod "0cc9a802-97f4-42c9-8c67-da1f30113dbf" (UID: "0cc9a802-97f4-42c9-8c67-da1f30113dbf"). InnerVolumeSpecName "kube-api-access-kjpw6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:37.458626 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:37.458598    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjpw6\" (UniqueName: \"kubernetes.io/projected/0cc9a802-97f4-42c9-8c67-da1f30113dbf-kube-api-access-kjpw6\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:56:38.059784 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.059693    2577 generic.go:356] "Generic (PLEG): container finished" podID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" containerID="57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4" exitCode=2
Sep 14 07:56:38.059784 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.059735    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-f2vtz"
Sep 14 07:56:38.060380 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.059776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-f2vtz" event={"ID":"0cc9a802-97f4-42c9-8c67-da1f30113dbf","Type":"ContainerDied","Data":"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"}
Sep 14 07:56:38.060380 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.059816    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-f2vtz" event={"ID":"0cc9a802-97f4-42c9-8c67-da1f30113dbf","Type":"ContainerDied","Data":"24037a346653bfa2fd178690a0d8351918e6be9567d7d5df5f5380378e42b60a"}
Sep 14 07:56:38.060380 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.059833    2577 scope.go:117] "RemoveContainer" containerID="57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"
Sep 14 07:56:38.061551 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.061524    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" event={"ID":"1b0fc740-bf99-4481-b1ca-6751cd586e42","Type":"ContainerStarted","Data":"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"}
Sep 14 07:56:38.061638 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.061536    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-55cc99f45f-fqxhh"
Sep 14 07:56:38.068235 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.068214    2577 scope.go:117] "RemoveContainer" containerID="57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"
Sep 14 07:56:38.068503 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:56:38.068486    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4\": container with ID starting with 57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4 not found: ID does not exist" containerID="57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"
Sep 14 07:56:38.068569 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.068509    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4"} err="failed to get container status \"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4\": rpc error: code = NotFound desc = could not find container \"57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4\": container with ID starting with 57d650f87d21d2bbaabebc17bc8fdc7309788538602800a75f8dcad90aa9f4e4 not found: ID does not exist"
Sep 14 07:56:38.090147 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.090127    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-55cc99f45f-fqxhh"]
Sep 14 07:56:38.094582 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.094561    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-55cc99f45f-fqxhh"]
Sep 14 07:56:38.126300 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.126253    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" podStartSLOduration=1.6792165479999999 podStartE2EDuration="2.126235856s" totalImagesPullingTime="447.019308ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:56:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:56:38.108541003 +0000 UTC m=+522.883039328" watchObservedRunningTime="2026-09-14 07:56:38.126235856 +0000 UTC m=+522.900734205"
Sep 14 07:56:38.127338 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.127321    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:38.132850 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.132822    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-f2vtz"]
Sep 14 07:56:38.136928 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.136891    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:38.137074 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.137055    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-lmfnx" podUID="5ea37f51-7963-4670-a58c-b933e2909a6d" containerName="authorino" containerID="cri-o://cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e" gracePeriod=30
Sep 14 07:56:38.388015 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.387991    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:38.467886 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.467851    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k2thw\" (UniqueName: \"kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw\") pod \"5ea37f51-7963-4670-a58c-b933e2909a6d\" (UID: \"5ea37f51-7963-4670-a58c-b933e2909a6d\") "
Sep 14 07:56:38.469981 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.469944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw" (OuterVolumeSpecName: "kube-api-access-k2thw") pod "5ea37f51-7963-4670-a58c-b933e2909a6d" (UID: "5ea37f51-7963-4670-a58c-b933e2909a6d"). InnerVolumeSpecName "kube-api-access-k2thw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 07:56:38.569278 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:38.569253    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k2thw\" (UniqueName: \"kubernetes.io/projected/5ea37f51-7963-4670-a58c-b933e2909a6d-kube-api-access-k2thw\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 07:56:39.068403 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.068370    2577 generic.go:356] "Generic (PLEG): container finished" podID="5ea37f51-7963-4670-a58c-b933e2909a6d" containerID="cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e" exitCode=0
Sep 14 07:56:39.068813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.068427    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-lmfnx"
Sep 14 07:56:39.068813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.068460    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lmfnx" event={"ID":"5ea37f51-7963-4670-a58c-b933e2909a6d","Type":"ContainerDied","Data":"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"}
Sep 14 07:56:39.068813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.068505    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-lmfnx" event={"ID":"5ea37f51-7963-4670-a58c-b933e2909a6d","Type":"ContainerDied","Data":"9b968a9d49963e28b57f6575f616860f2f4beccc93b6d6096527af501a1f57b2"}
Sep 14 07:56:39.068813 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.068527    2577 scope.go:117] "RemoveContainer" containerID="cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"
Sep 14 07:56:39.077086 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.077066    2577 scope.go:117] "RemoveContainer" containerID="cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"
Sep 14 07:56:39.077357 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:56:39.077340    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e\": container with ID starting with cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e not found: ID does not exist" containerID="cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"
Sep 14 07:56:39.077408 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.077363    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e"} err="failed to get container status \"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e\": rpc error: code = NotFound desc = could not find container \"cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e\": container with ID starting with cc0623914d039dfdd932678b3e3fc24c6efb48fbe7114829fa899f3a789c897e not found: ID does not exist"
Sep 14 07:56:39.090584 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.090562    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:39.096409 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.096390    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-lmfnx"]
Sep 14 07:56:39.759497 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.759463    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" path="/var/lib/kubelet/pods/0cc9a802-97f4-42c9-8c67-da1f30113dbf/volumes"
Sep 14 07:56:39.759807 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.759793    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5ea37f51-7963-4670-a58c-b933e2909a6d" path="/var/lib/kubelet/pods/5ea37f51-7963-4670-a58c-b933e2909a6d/volumes"
Sep 14 07:56:39.760128 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:56:39.760116    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="876d7d7f-a79e-4453-b495-0960fb5d0e26" path="/var/lib/kubelet/pods/876d7d7f-a79e-4453-b495-0960fb5d0e26/volumes"
Sep 14 07:57:21.090187 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090151    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090475    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ea37f51-7963-4670-a58c-b933e2909a6d" containerName="authorino"
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090486    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ea37f51-7963-4670-a58c-b933e2909a6d" containerName="authorino"
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090503    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" containerName="authorino"
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090508    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" containerName="authorino"
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090562    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5ea37f51-7963-4670-a58c-b933e2909a6d" containerName="authorino"
Sep 14 07:57:21.090654 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.090569    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cc9a802-97f4-42c9-8c67-da1f30113dbf" containerName="authorino"
Sep 14 07:57:21.093178 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.093162    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.105933 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.105898    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 14 07:57:21.106049 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.105901    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 14 07:57:21.106262 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.106239    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 14 07:57:21.107746 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.107726    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-g2t2q\""
Sep 14 07:57:21.107838 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.107748    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 14 07:57:21.109889 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.109869    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 07:57:21.131500 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.131472    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.131607 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.131506    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.131607 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.131576    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xqqr\" (UniqueName: \"kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.131677 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.131639    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.232252 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.232219    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.232408 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.232261    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.232408 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.232279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.232408 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.232325    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xqqr\" (UniqueName: \"kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.232408 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:57:21.232375    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 14 07:57:21.232570 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:57:21.232453    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found
Sep 14 07:57:21.232570 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:57:21.232462    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs podName:cbc1bdc9-ed11-411b-9057-e08b7784e77d nodeName:}" failed. No retries permitted until 2026-09-14 07:57:21.732439425 +0000 UTC m=+566.506937755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs") pod "maas-api-765c9ffcd4-f79dv" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d") : secret "maas-api-metrics-tls" not found
Sep 14 07:57:21.232570 ip-10-0-128-30 kubenswrapper[2577]: E0914 07:57:21.232527    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls podName:cbc1bdc9-ed11-411b-9057-e08b7784e77d nodeName:}" failed. No retries permitted until 2026-09-14 07:57:21.73250927 +0000 UTC m=+566.507007589 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls") pod "maas-api-765c9ffcd4-f79dv" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d") : secret "maas-api-serving-cert" not found
Sep 14 07:57:21.232890 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.232872    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.246469 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.246448    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xqqr\" (UniqueName: \"kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.726775 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.726737    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"]
Sep 14 07:57:21.730151 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.730129    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.733051 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.733033    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 14 07:57:21.737518 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.737484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.737629 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.737611    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.741327 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.740544    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.741327 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.740632    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") pod \"maas-api-765c9ffcd4-f79dv\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") " pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:21.741553 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.741532    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"]
Sep 14 07:57:21.839101 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.839062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62befff0-bbee-442a-9563-53438eeae0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.839272 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.839169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h5bq\" (UniqueName: \"kubernetes.io/projected/62befff0-bbee-442a-9563-53438eeae0e9-kube-api-access-2h5bq\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.939842 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.939807    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62befff0-bbee-442a-9563-53438eeae0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.940019 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.939907    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h5bq\" (UniqueName: \"kubernetes.io/projected/62befff0-bbee-442a-9563-53438eeae0e9-kube-api-access-2h5bq\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.940642 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.940621    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/62befff0-bbee-442a-9563-53438eeae0e9-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:21.949501 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:21.949469    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h5bq\" (UniqueName: \"kubernetes.io/projected/62befff0-bbee-442a-9563-53438eeae0e9-kube-api-access-2h5bq\") pod \"payload-pre-processing-59fbc47bf8-x4j4k\" (UID: \"62befff0-bbee-442a-9563-53438eeae0e9\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:22.003324 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:22.003217    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:22.043188 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:22.043149    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:22.292064 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:22.292035    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 07:57:22.293182 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:57:22.293158    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbc1bdc9_ed11_411b_9057_e08b7784e77d.slice/crio-eb589366f7c9ddcc45256e06f5543e6037b2708e07b7b34513c3936c6586d6ed WatchSource:0}: Error finding container eb589366f7c9ddcc45256e06f5543e6037b2708e07b7b34513c3936c6586d6ed: Status 404 returned error can't find the container with id eb589366f7c9ddcc45256e06f5543e6037b2708e07b7b34513c3936c6586d6ed
Sep 14 07:57:22.338267 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:22.338244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"]
Sep 14 07:57:22.340668 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:57:22.340644    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62befff0_bbee_442a_9563_53438eeae0e9.slice/crio-39c8d005dc0d14c34c9c364629eec8bfc0e20de14d0baa7e2266e1c62295048a WatchSource:0}: Error finding container 39c8d005dc0d14c34c9c364629eec8bfc0e20de14d0baa7e2266e1c62295048a: Status 404 returned error can't find the container with id 39c8d005dc0d14c34c9c364629eec8bfc0e20de14d0baa7e2266e1c62295048a
Sep 14 07:57:23.279297 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:23.279259    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k" event={"ID":"62befff0-bbee-442a-9563-53438eeae0e9","Type":"ContainerStarted","Data":"39c8d005dc0d14c34c9c364629eec8bfc0e20de14d0baa7e2266e1c62295048a"}
Sep 14 07:57:23.280720 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:23.280684    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" event={"ID":"cbc1bdc9-ed11-411b-9057-e08b7784e77d","Type":"ContainerStarted","Data":"eb589366f7c9ddcc45256e06f5543e6037b2708e07b7b34513c3936c6586d6ed"}
Sep 14 07:57:24.288576 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:24.288541    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k" event={"ID":"62befff0-bbee-442a-9563-53438eeae0e9","Type":"ContainerStarted","Data":"d27b3e356d58fadfed0f93ab5a3156e67346d3c3fc2131d4b4f472006b571c91"}
Sep 14 07:57:24.289060 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:24.288637    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:24.310056 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:24.310005    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k" podStartSLOduration=1.808961473 podStartE2EDuration="3.3099837s" totalImagesPullingTime="1.501022227s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:24.307340644 +0000 UTC m=+569.081838984" watchObservedRunningTime="2026-09-14 07:57:24.3099837 +0000 UTC m=+569.084482037"
Sep 14 07:57:25.295094 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:25.295060    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" event={"ID":"cbc1bdc9-ed11-411b-9057-e08b7784e77d","Type":"ContainerStarted","Data":"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"}
Sep 14 07:57:25.295535 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:25.295244    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:25.315351 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:25.315300    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" podStartSLOduration=1.46275385 podStartE2EDuration="4.315282602s" totalImagesPullingTime="2.852528752s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:57:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:57:25.314150313 +0000 UTC m=+570.088648648" watchObservedRunningTime="2026-09-14 07:57:25.315282602 +0000 UTC m=+570.089780941"
Sep 14 07:57:31.303816 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:31.303788    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 07:57:35.297173 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:35.297145    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-x4j4k"
Sep 14 07:57:55.693280 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:55.693253    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:57:55.696304 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:57:55.696283    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 07:58:19.405684 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.405599    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"]
Sep 14 07:58:19.409010 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.408992    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.412960 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.412942    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-7j9jg\""
Sep 14 07:58:19.412960 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.412950    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 14 07:58:19.413114 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.412952    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 14 07:58:19.413114 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.412952    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 14 07:58:19.419964 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.419942    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"]
Sep 14 07:58:19.544251 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.544425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.544425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqftw\" (UniqueName: \"kubernetes.io/projected/ae8eacff-8d1b-40fa-b795-464ceb49179f-kube-api-access-nqftw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.544425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544312    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.544425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae8eacff-8d1b-40fa-b795-464ceb49179f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.544425 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.544411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645719 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645687    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645719 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645724    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae8eacff-8d1b-40fa-b795-464ceb49179f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645943 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645943 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645780    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645943 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645803    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.645943 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.645822    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqftw\" (UniqueName: \"kubernetes.io/projected/ae8eacff-8d1b-40fa-b795-464ceb49179f-kube-api-access-nqftw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.646177 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.646152    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.646237 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.646201    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.646277 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.646249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.648038 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.648018    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae8eacff-8d1b-40fa-b795-464ceb49179f-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.648230 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.648211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae8eacff-8d1b-40fa-b795-464ceb49179f-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.656429 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.656377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqftw\" (UniqueName: \"kubernetes.io/projected/ae8eacff-8d1b-40fa-b795-464ceb49179f-kube-api-access-nqftw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nnt76\" (UID: \"ae8eacff-8d1b-40fa-b795-464ceb49179f\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.719055 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.719030    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:19.990793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.990765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"]
Sep 14 07:58:19.992812 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:58:19.992783    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8eacff_8d1b_40fa_b795_464ceb49179f.slice/crio-14d82a9cc1a11a57bd09392e00d08222cfed2f48cac383d7758794c4daa6725e WatchSource:0}: Error finding container 14d82a9cc1a11a57bd09392e00d08222cfed2f48cac383d7758794c4daa6725e: Status 404 returned error can't find the container with id 14d82a9cc1a11a57bd09392e00d08222cfed2f48cac383d7758794c4daa6725e
Sep 14 07:58:19.994226 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:19.994208    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 07:58:20.550782 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:20.550748    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76" event={"ID":"ae8eacff-8d1b-40fa-b795-464ceb49179f","Type":"ContainerStarted","Data":"14d82a9cc1a11a57bd09392e00d08222cfed2f48cac383d7758794c4daa6725e"}
Sep 14 07:58:21.557783 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:21.557747    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76" event={"ID":"ae8eacff-8d1b-40fa-b795-464ceb49179f","Type":"ContainerStarted","Data":"299073e4c1b9985a8363b7dc8c1f2c546d0912445f027239ac0ae80694e29ff8"}
Sep 14 07:58:21.581562 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:21.581514    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76" podStartSLOduration=1.645757307 podStartE2EDuration="2.581495204s" totalImagesPullingTime="935.737897ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:58:21.58047951 +0000 UTC m=+626.354977848" watchObservedRunningTime="2026-09-14 07:58:21.581495204 +0000 UTC m=+626.355993538"
Sep 14 07:58:29.719793 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:29.719760    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:29.720225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:29.719845    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:29.732675 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:29.732646    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:30.612093 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:30.612066    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nnt76"
Sep 14 07:58:59.896312 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.896279    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"]
Sep 14 07:58:59.904637 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.903131    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.906788 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.906766    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 14 07:58:59.908186 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.908159    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"]
Sep 14 07:58:59.980868 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.980833    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.980868 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.980868    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf48g\" (UniqueName: \"kubernetes.io/projected/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-kube-api-access-tf48g\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.981081 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.980993    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.981081 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.981018    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.981081 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.981062    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:58:59.981225 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:58:59.981099    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082507 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082454    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082507 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082523    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082768 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082560    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082768 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082576    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf48g\" (UniqueName: \"kubernetes.io/projected/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-kube-api-access-tf48g\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082768 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082615    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.082768 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.082632    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.083071 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.083047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.083137 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.083070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.083137 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.083118    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.084838 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.084817    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.085001 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.084984    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.091113 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.091087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf48g\" (UniqueName: \"kubernetes.io/projected/10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea-kube-api-access-tf48g\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft\" (UID: \"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.214161 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.214118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:00.500286 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.500260    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"]
Sep 14 07:59:00.503048 ip-10-0-128-30 kubenswrapper[2577]: W0914 07:59:00.503024    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10143ce2_e149_4b51_a9fd_6eb6c7e0a2ea.slice/crio-9bc8ffa211b2055061a966b4f764524b7ab9b4fcb7a1afcb2d6ceaa69ff8c2bf WatchSource:0}: Error finding container 9bc8ffa211b2055061a966b4f764524b7ab9b4fcb7a1afcb2d6ceaa69ff8c2bf: Status 404 returned error can't find the container with id 9bc8ffa211b2055061a966b4f764524b7ab9b4fcb7a1afcb2d6ceaa69ff8c2bf
Sep 14 07:59:00.738622 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:00.738597    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft" event={"ID":"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea","Type":"ContainerStarted","Data":"9bc8ffa211b2055061a966b4f764524b7ab9b4fcb7a1afcb2d6ceaa69ff8c2bf"}
Sep 14 07:59:01.744341 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:01.744300    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft" event={"ID":"10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea","Type":"ContainerStarted","Data":"d96f00376ab8a1ee2c021c34beff236de3ddfe61c76a2f92e804dad51119ae0d"}
Sep 14 07:59:01.763856 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:01.763803    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft" podStartSLOduration=2.569985187 podStartE2EDuration="2.763786562s" totalImagesPullingTime="193.801375ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 07:58:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 07:59:01.762362306 +0000 UTC m=+666.536860653" watchObservedRunningTime="2026-09-14 07:59:01.763786562 +0000 UTC m=+666.538284897"
Sep 14 07:59:10.214856 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:10.214820    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:10.215268 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:10.214974    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:10.227613 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:10.227587    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 07:59:10.794862 ip-10-0-128-30 kubenswrapper[2577]: I0914 07:59:10.794835    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft"
Sep 14 08:00:11.626958 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.626861    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76dd588448-bjbvq"]
Sep 14 08:00:11.630256 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.630238    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.638602 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.638579    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76dd588448-bjbvq"]
Sep 14 08:00:11.681580 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.681546    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8wqd\" (UniqueName: \"kubernetes.io/projected/a53d937d-5ca1-49f3-8312-27fa572b8b83-kube-api-access-r8wqd\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.681712 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.681585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a53d937d-5ca1-49f3-8312-27fa572b8b83-tls-cert\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.782729 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.782698    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8wqd\" (UniqueName: \"kubernetes.io/projected/a53d937d-5ca1-49f3-8312-27fa572b8b83-kube-api-access-r8wqd\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.782873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.782738    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a53d937d-5ca1-49f3-8312-27fa572b8b83-tls-cert\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.785342 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.785321    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a53d937d-5ca1-49f3-8312-27fa572b8b83-tls-cert\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.790863 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.790837    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8wqd\" (UniqueName: \"kubernetes.io/projected/a53d937d-5ca1-49f3-8312-27fa572b8b83-kube-api-access-r8wqd\") pod \"authorino-76dd588448-bjbvq\" (UID: \"a53d937d-5ca1-49f3-8312-27fa572b8b83\") " pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:11.939544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:11.939514    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76dd588448-bjbvq"
Sep 14 08:00:12.211239 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:12.211155    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76dd588448-bjbvq"]
Sep 14 08:00:12.213477 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:12.213452    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda53d937d_5ca1_49f3_8312_27fa572b8b83.slice/crio-d234c404a4b223a5a3e4c1d828fc0b4c5839eb1118dbc94e2250d29d6ff007ee WatchSource:0}: Error finding container d234c404a4b223a5a3e4c1d828fc0b4c5839eb1118dbc94e2250d29d6ff007ee: Status 404 returned error can't find the container with id d234c404a4b223a5a3e4c1d828fc0b4c5839eb1118dbc94e2250d29d6ff007ee
Sep 14 08:00:13.072724 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.072685    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76dd588448-bjbvq" event={"ID":"a53d937d-5ca1-49f3-8312-27fa572b8b83","Type":"ContainerStarted","Data":"74b43a4a1e6f174f48fe0c314c7de0cb54c3c87867b2e55bb66cd8fa05d5571d"}
Sep 14 08:00:13.072724 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.072730    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-76dd588448-bjbvq" event={"ID":"a53d937d-5ca1-49f3-8312-27fa572b8b83","Type":"ContainerStarted","Data":"d234c404a4b223a5a3e4c1d828fc0b4c5839eb1118dbc94e2250d29d6ff007ee"}
Sep 14 08:00:13.090835 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.090773    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-76dd588448-bjbvq" podStartSLOduration=1.3903957 podStartE2EDuration="2.090751637s" totalImagesPullingTime="700.355937ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:13.089781429 +0000 UTC m=+737.864279765" watchObservedRunningTime="2026-09-14 08:00:13.090751637 +0000 UTC m=+737.865249973"
Sep 14 08:00:13.119450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.119416    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 08:00:13.119643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.119621    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" podUID="1b0fc740-bf99-4481-b1ca-6751cd586e42" containerName="authorino" containerID="cri-o://c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9" gracePeriod=30
Sep 14 08:00:13.365109 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.365086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 08:00:13.498589 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.498508    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wm666\" (UniqueName: \"kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666\") pod \"1b0fc740-bf99-4481-b1ca-6751cd586e42\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") "
Sep 14 08:00:13.498754 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.498634    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert\") pod \"1b0fc740-bf99-4481-b1ca-6751cd586e42\" (UID: \"1b0fc740-bf99-4481-b1ca-6751cd586e42\") "
Sep 14 08:00:13.500685 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.500649    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666" (OuterVolumeSpecName: "kube-api-access-wm666") pod "1b0fc740-bf99-4481-b1ca-6751cd586e42" (UID: "1b0fc740-bf99-4481-b1ca-6751cd586e42"). InnerVolumeSpecName "kube-api-access-wm666". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:00:13.509800 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.509767    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "1b0fc740-bf99-4481-b1ca-6751cd586e42" (UID: "1b0fc740-bf99-4481-b1ca-6751cd586e42"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:00:13.599338 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.599303    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm666\" (UniqueName: \"kubernetes.io/projected/1b0fc740-bf99-4481-b1ca-6751cd586e42-kube-api-access-wm666\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:00:13.599338 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:13.599332    2577 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/1b0fc740-bf99-4481-b1ca-6751cd586e42-tls-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:00:14.078569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.078476    2577 generic.go:356] "Generic (PLEG): container finished" podID="1b0fc740-bf99-4481-b1ca-6751cd586e42" containerID="c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9" exitCode=0
Sep 14 08:00:14.078569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.078517    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7568d7cd6f-78jbl"
Sep 14 08:00:14.079123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.078562    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" event={"ID":"1b0fc740-bf99-4481-b1ca-6751cd586e42","Type":"ContainerDied","Data":"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"}
Sep 14 08:00:14.079123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.078607    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7568d7cd6f-78jbl" event={"ID":"1b0fc740-bf99-4481-b1ca-6751cd586e42","Type":"ContainerDied","Data":"57c94b081a29f158eb76e5e55833f15767f81ce29bf332cc1a3759865a66b086"}
Sep 14 08:00:14.079123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.078623    2577 scope.go:117] "RemoveContainer" containerID="c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"
Sep 14 08:00:14.087517 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.087493    2577 scope.go:117] "RemoveContainer" containerID="c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"
Sep 14 08:00:14.087832 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:14.087805    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9\": container with ID starting with c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9 not found: ID does not exist" containerID="c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"
Sep 14 08:00:14.087952 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.087839    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9"} err="failed to get container status \"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9\": rpc error: code = NotFound desc = could not find container \"c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9\": container with ID starting with c55ad38d4c08ba7f899b188d3950a4431eba40339fb8a14454fc3a0283262aa9 not found: ID does not exist"
Sep 14 08:00:14.097384 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.097360    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 08:00:14.101154 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:14.101133    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7568d7cd6f-78jbl"]
Sep 14 08:00:15.760425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:15.760392    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b0fc740-bf99-4481-b1ca-6751cd586e42" path="/var/lib/kubelet/pods/1b0fc740-bf99-4481-b1ca-6751cd586e42/volumes"
Sep 14 08:00:30.613895 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.613865    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:00:30.614299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.614278    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b0fc740-bf99-4481-b1ca-6751cd586e42" containerName="authorino"
Sep 14 08:00:30.614299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.614291    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b0fc740-bf99-4481-b1ca-6751cd586e42" containerName="authorino"
Sep 14 08:00:30.614386 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.614368    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b0fc740-bf99-4481-b1ca-6751cd586e42" containerName="authorino"
Sep 14 08:00:30.617653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.617633    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.620565 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.620532    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:00:30.620565 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.620532    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-b69a34-openshift-default-dockercfg-8rkvw\""
Sep 14 08:00:30.630248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.630223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:00:30.753881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.753852    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.753881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.753884    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.753911    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjpnh\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.753961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.754008    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.754030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.754084    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.754152    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.754255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.754177    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.854963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.854902    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.854963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.854958    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855190 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.854981    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjpnh\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855190 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855007    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855190 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855046    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855190 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855109    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855389 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855329    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855389 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855494 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855435    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855503    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855642 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855721 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855700    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855773 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855757    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.855841 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.855821    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.857580 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.857560    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.857786 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.857770    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.874153 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.874102    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.876319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.876239    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjpnh\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh\") pod \"e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") " pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:30.929079 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:30.929056    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:31.207943 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:31.207891    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:00:31.210204 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:31.210178    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76293e7e_c869_4277_812e_4c3439552980.slice/crio-81d4627b01f19143d469086a5d201b619deb9f8bac748e23531041f86d296264 WatchSource:0}: Error finding container 81d4627b01f19143d469086a5d201b619deb9f8bac748e23531041f86d296264: Status 404 returned error can't find the container with id 81d4627b01f19143d469086a5d201b619deb9f8bac748e23531041f86d296264
Sep 14 08:00:31.212555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:31.212518    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:31.212630 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:31.212587    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:31.212630 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:31.212622    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:32.167682 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:32.167650    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerStarted","Data":"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"}
Sep 14 08:00:32.168097 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:32.167699    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerStarted","Data":"81d4627b01f19143d469086a5d201b619deb9f8bac748e23531041f86d296264"}
Sep 14 08:00:32.211939 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:32.211866    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podStartSLOduration=2.211849597 podStartE2EDuration="2.211849597s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:32.210673417 +0000 UTC m=+756.985171755" watchObservedRunningTime="2026-09-14 08:00:32.211849597 +0000 UTC m=+756.986347933"
Sep 14 08:00:32.929654 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:32.929616    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:32.934547 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:32.934523    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:33.172546 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:33.172515    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:33.173420 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:33.173401    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:35.640051 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.640011    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:00:35.643490 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.643470    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.649008 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.648983    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w4-b69a34-serving-cert\""
Sep 14 08:00:35.677005 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.676972    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:00:35.802170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.802136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.802330 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.802185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.802330 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.802240    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9csc\" (UniqueName: \"kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.802406 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.802360    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.903257 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.903227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.903411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.903334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.903411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.903378    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.903411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.903395    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9csc\" (UniqueName: \"kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.904088 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.904064    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.905750 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.905717    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.905829 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.905795    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.918435 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.918413    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9csc\" (UniqueName: \"kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc\") pod \"maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:35.954277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:35.954248    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:36.230859 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.230836    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:00:36.232400 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:36.232376    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98e8625a_d408_41b6_9d45_f2bd151f6b32.slice/crio-ce5e39445b617f62f702e254c7bf05ef39380c8af9ea65d6623a6d4028183e4a WatchSource:0}: Error finding container ce5e39445b617f62f702e254c7bf05ef39380c8af9ea65d6623a6d4028183e4a: Status 404 returned error can't find the container with id ce5e39445b617f62f702e254c7bf05ef39380c8af9ea65d6623a6d4028183e4a
Sep 14 08:00:36.636869 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.636801    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7873-fd5113"]
Sep 14 08:00:36.641369 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.641345    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:36.645086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.644525    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fvblv\""
Sep 14 08:00:36.645896 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.645871    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-fd5113"]
Sep 14 08:00:36.804458 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.804429    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:00:36.807944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.807906    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:36.810553 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.810529    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-814672-serving-cert\""
Sep 14 08:00:36.810768 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.810749    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv54s\" (UniqueName: \"kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s\") pod \"test-curl-7873-fd5113\" (UID: \"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2\") " pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:36.818035 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.818011    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:00:36.911818 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.911778    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2hp2\" (UniqueName: \"kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:36.912016 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.911876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:36.912016 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.911994    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv54s\" (UniqueName: \"kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s\") pod \"test-curl-7873-fd5113\" (UID: \"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2\") " pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:36.912152 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.912023    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:36.912152 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.912102    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:36.920951 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.920927    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv54s\" (UniqueName: \"kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s\") pod \"test-curl-7873-fd5113\" (UID: \"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2\") " pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:36.952583 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:36.952552    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:37.013534 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.013503    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j2hp2\" (UniqueName: \"kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.013655 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.013558    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.013655 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.013601    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.013655 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.013621    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.014465 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.014444    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.016086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.016068    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.016579 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.016557    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.033444 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.033418    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2hp2\" (UniqueName: \"kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2\") pod \"maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.121709 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.121665    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:37.202209 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.202160    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" event={"ID":"98e8625a-d408-41b6-9d45-f2bd151f6b32","Type":"ContainerStarted","Data":"ce5e39445b617f62f702e254c7bf05ef39380c8af9ea65d6623a6d4028183e4a"}
Sep 14 08:00:37.229208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.229181    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7873-fd5113"]
Sep 14 08:00:37.235059 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:37.235035    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd00ec2a4_9be7_4e7f_9d37_8c120a73aaa2.slice/crio-1efa10f26c47e22978a26461bdc4131eaa852bab15dcaeec44056801be0423ad WatchSource:0}: Error finding container 1efa10f26c47e22978a26461bdc4131eaa852bab15dcaeec44056801be0423ad: Status 404 returned error can't find the container with id 1efa10f26c47e22978a26461bdc4131eaa852bab15dcaeec44056801be0423ad
Sep 14 08:00:37.414549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.414525    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:00:37.416862 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:37.416836    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c563cb6_bc68_4755_9c36_c712d5e36f2d.slice/crio-f9bf2b5b3b4568a158026f086adf99acabdff9732e74f06914da69f225974e7b WatchSource:0}: Error finding container f9bf2b5b3b4568a158026f086adf99acabdff9732e74f06914da69f225974e7b: Status 404 returned error can't find the container with id f9bf2b5b3b4568a158026f086adf99acabdff9732e74f06914da69f225974e7b
Sep 14 08:00:37.987094 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.987060    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:00:37.990761 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.990733    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:37.997584 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:37.997562    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-3a842e-serving-cert\""
Sep 14 08:00:38.007477 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.007450    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:00:38.123509 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.123424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.123509 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.123461    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.123509 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.123488    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp6z2\" (UniqueName: \"kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.123746 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.123575    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.208749 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.208714    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" event={"ID":"98e8625a-d408-41b6-9d45-f2bd151f6b32","Type":"ContainerStarted","Data":"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"}
Sep 14 08:00:38.208971 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.208911    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:38.210234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.210213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-fd5113" event={"ID":"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2","Type":"ContainerStarted","Data":"a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608"}
Sep 14 08:00:38.210328 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.210241    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7873-fd5113" event={"ID":"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2","Type":"ContainerStarted","Data":"1efa10f26c47e22978a26461bdc4131eaa852bab15dcaeec44056801be0423ad"}
Sep 14 08:00:38.211573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.211553    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" event={"ID":"1c563cb6-bc68-4755-9c36-c712d5e36f2d","Type":"ContainerStarted","Data":"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"}
Sep 14 08:00:38.211637 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.211589    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" event={"ID":"1c563cb6-bc68-4755-9c36-c712d5e36f2d","Type":"ContainerStarted","Data":"f9bf2b5b3b4568a158026f086adf99acabdff9732e74f06914da69f225974e7b"}
Sep 14 08:00:38.211637 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.211620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:38.224663 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.224640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.224735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.224671    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.224735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.224694    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp6z2\" (UniqueName: \"kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.224735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.224726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.224856 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:38.224775    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-serving-cert: secret "maas-api-e2e-inf-a-3a842e-serving-cert" not found
Sep 14 08:00:38.224856 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:38.224828    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls podName:3cf6ebd1-4629-4cce-843a-4f668b8ad233 nodeName:}" failed. No retries permitted until 2026-09-14 08:00:38.724812315 +0000 UTC m=+763.499310646 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls") pod "maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" (UID: "3cf6ebd1-4629-4cce-843a-4f668b8ad233") : secret "maas-api-e2e-inf-a-3a842e-serving-cert" not found
Sep 14 08:00:38.225390 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.225372    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.227230 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.227211    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.246628 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.246603    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp6z2\" (UniqueName: \"kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.254872 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.254829    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" podStartSLOduration=2.278724877 podStartE2EDuration="3.254813046s" totalImagesPullingTime="976.088169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:38.232520532 +0000 UTC m=+763.007018868" watchObservedRunningTime="2026-09-14 08:00:38.254813046 +0000 UTC m=+763.029311381"
Sep 14 08:00:38.256588 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.256542    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7873-fd5113" podStartSLOduration=1.883998562 podStartE2EDuration="2.256528766s" totalImagesPullingTime="372.530204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:38.25400023 +0000 UTC m=+763.028498565" watchObservedRunningTime="2026-09-14 08:00:38.256528766 +0000 UTC m=+763.031027101"
Sep 14 08:00:38.276592 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.276555    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" podStartSLOduration=1.900478672 podStartE2EDuration="2.276543206s" totalImagesPullingTime="376.064534ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:38.274694029 +0000 UTC m=+763.049192364" watchObservedRunningTime="2026-09-14 08:00:38.276543206 +0000 UTC m=+763.051041540"
Sep 14 08:00:38.728927 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.728836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.731049 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.731020    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") pod \"maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:38.891207 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.891174    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7873-fd5113"]
Sep 14 08:00:38.898250 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.898221    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7873-fd5113"]
Sep 14 08:00:38.901644 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:38.901617    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:39.177368 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:39.177343    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:00:39.179291 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:39.179264    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cf6ebd1_4629_4cce_843a_4f668b8ad233.slice/crio-325f9328441287a14db7283874569a37deaffc03c80aef1bb9cdc39b83a904c1 WatchSource:0}: Error finding container 325f9328441287a14db7283874569a37deaffc03c80aef1bb9cdc39b83a904c1: Status 404 returned error can't find the container with id 325f9328441287a14db7283874569a37deaffc03c80aef1bb9cdc39b83a904c1
Sep 14 08:00:39.218407 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:39.218374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" event={"ID":"3cf6ebd1-4629-4cce-843a-4f668b8ad233","Type":"ContainerStarted","Data":"325f9328441287a14db7283874569a37deaffc03c80aef1bb9cdc39b83a904c1"}
Sep 14 08:00:39.218595 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:39.218571    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7873-fd5113" container="test-curl-7873-fd5113"
Sep 14 08:00:40.225012 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:40.224962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" event={"ID":"3cf6ebd1-4629-4cce-843a-4f668b8ad233","Type":"ContainerStarted","Data":"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"}
Sep 14 08:00:40.225450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:40.225012    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7873-fd5113" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" containerName="test-curl-7873-fd5113" containerID="cri-o://a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608" gracePeriod=2
Sep 14 08:00:40.225450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:40.225255    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:40.246358 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:40.246312    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" podStartSLOduration=2.767863603 podStartE2EDuration="3.246296248s" totalImagesPullingTime="478.432645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:40.243767886 +0000 UTC m=+765.018266222" watchObservedRunningTime="2026-09-14 08:00:40.246296248 +0000 UTC m=+765.020794583"
Sep 14 08:00:42.367406 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:42.367379    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:42.467134 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:42.467105    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv54s\" (UniqueName: \"kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s\") pod \"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2\" (UID: \"d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2\") "
Sep 14 08:00:42.469287 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:42.469254    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s" (OuterVolumeSpecName: "kube-api-access-cv54s") pod "d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" (UID: "d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2"). InnerVolumeSpecName "kube-api-access-cv54s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:00:42.567806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:42.567726    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv54s\" (UniqueName: \"kubernetes.io/projected/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2-kube-api-access-cv54s\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:00:43.242167 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.242134    2577 generic.go:356] "Generic (PLEG): container finished" podID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" containerID="a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608" exitCode=137
Sep 14 08:00:43.242351 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.242172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:43.242351 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.242239    2577 scope.go:117] "RemoveContainer" containerID="a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608"
Sep 14 08:00:43.252082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.252065    2577 scope.go:117] "RemoveContainer" containerID="a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608"
Sep 14 08:00:43.252381 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:43.252358    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608\": container with ID starting with a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608 not found: ID does not exist" containerID="a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608"
Sep 14 08:00:43.252455 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.252390    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608"} err="failed to get container status \"a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608\": rpc error: code = NotFound desc = could not find container \"a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608\": container with ID starting with a4805006a81c107a822f88ee20184dd67da6cc3e67e94c59c57d05e8bb972608 not found: ID does not exist"
Sep 14 08:00:43.253835 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:43.253806    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7873-fd5113\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" pod="opendatahub/test-curl-7873-fd5113"
Sep 14 08:00:43.760641 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.760602    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" path="/var/lib/kubelet/pods/d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2/volumes"
Sep 14 08:00:43.956938 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.956507    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:00:43.959433 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.958009    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" containerName="test-curl-7873-fd5113"
Sep 14 08:00:43.959433 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.958034    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" containerName="test-curl-7873-fd5113"
Sep 14 08:00:43.959433 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.958156    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d00ec2a4-9be7-4e7f-9d37-8c120a73aaa2" containerName="test-curl-7873-fd5113"
Sep 14 08:00:43.965729 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.965704    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:43.974360 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.974144    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-22b517-serving-cert\""
Sep 14 08:00:43.997357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:43.995685    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:00:44.078991 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.078880    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.078991 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.078954    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.078991 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.078989    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.079267 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.079198    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9t7d\" (UniqueName: \"kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.179787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.179744    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.179787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.179792    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.180074 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:44.179912    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-22b517-serving-cert: secret "maas-api-e2e-autores-22b517-serving-cert" not found
Sep 14 08:00:44.180074 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:00:44.180013    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls podName:ade7ae9d-8af0-454c-94ce-f4ff80d69e75 nodeName:}" failed. No retries permitted until 2026-09-14 08:00:44.679990495 +0000 UTC m=+769.454488808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls") pod "maas-api-e2e-autores-22b517-9f6b6568c-nfb28" (UID: "ade7ae9d-8af0-454c-94ce-f4ff80d69e75") : secret "maas-api-e2e-autores-22b517-serving-cert" not found
Sep 14 08:00:44.180074 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.179912    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.180264 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.180120    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9t7d\" (UniqueName: \"kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.180532 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.180512    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.183014 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.182986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.191792 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.191564    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9t7d\" (UniqueName: \"kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.224290 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.224251    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:00:44.224725 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.224701    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:00:44.683836 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.683801    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.686244 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.686217    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") pod \"maas-api-e2e-autores-22b517-9f6b6568c-nfb28\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:44.907667 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:44.907635    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:45.187321 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:45.187294    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:00:45.189363 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:45.189338    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podade7ae9d_8af0_454c_94ce_f4ff80d69e75.slice/crio-8e74e7ce8dd151835526fe638085634e9bdb2f46f9fba5a3b61f9e5eb6666760 WatchSource:0}: Error finding container 8e74e7ce8dd151835526fe638085634e9bdb2f46f9fba5a3b61f9e5eb6666760: Status 404 returned error can't find the container with id 8e74e7ce8dd151835526fe638085634e9bdb2f46f9fba5a3b61f9e5eb6666760
Sep 14 08:00:45.256406 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:45.256321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" event={"ID":"ade7ae9d-8af0-454c-94ce-f4ff80d69e75","Type":"ContainerStarted","Data":"8e74e7ce8dd151835526fe638085634e9bdb2f46f9fba5a3b61f9e5eb6666760"}
Sep 14 08:00:46.236174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:46.236140    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:00:46.264075 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:46.264044    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" event={"ID":"ade7ae9d-8af0-454c-94ce-f4ff80d69e75","Type":"ContainerStarted","Data":"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"}
Sep 14 08:00:46.264255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:46.264147    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:46.295573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:46.295063    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" podStartSLOduration=2.45188292 podStartE2EDuration="3.295038276s" totalImagesPullingTime="843.155356ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:46.294234684 +0000 UTC m=+771.068733025" watchObservedRunningTime="2026-09-14 08:00:46.295038276 +0000 UTC m=+771.069536613"
Sep 14 08:00:48.414796 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.414757    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:00:48.418856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.418834    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.421679 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.421661    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-a9d21f4e-serving-cert\""
Sep 14 08:00:48.430251 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.430223    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:00:48.528115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.528070    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.528297 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.528139    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.528297 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.528224    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m29x\" (UniqueName: \"kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.528297 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.528253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.629605 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.629568    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.629788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.629649    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m29x\" (UniqueName: \"kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.629788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.629669    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.629788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.629709    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.630481 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.630439    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.632280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.632258    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.632864 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.632840    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.639371 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.639351    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m29x\" (UniqueName: \"kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x\") pod \"maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:48.731054 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:48.730951    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:49.033472 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.033388    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:00:49.204786 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.204747    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:00:49.209425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.209407    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.212301 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.212280    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-a9d21f4e-dockercfg-bgkh9\""
Sep 14 08:00:49.212524 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.212502    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-a9d21f4e\""
Sep 14 08:00:49.224960 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.224937    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:00:49.236905 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.236875    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbqn\" (UniqueName: \"kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.237037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.236969    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.302938 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.302826    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" event={"ID":"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9","Type":"ContainerStarted","Data":"220fa09cd27025c215b7b2239cabd9c1c0998c74075d71c710c4fa10874a7c92"}
Sep 14 08:00:49.338515 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.338478    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.338713 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.338590    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbqn\" (UniqueName: \"kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.339436 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.339408    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.347905 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.347884    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbqn\" (UniqueName: \"kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn\") pod \"payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.520193 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.520155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:49.799483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:49.796510    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:00:49.800302 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:00:49.800275    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e76cec_8c89_43a0_938d_7a8c74194668.slice/crio-d1313b9f28ddbc2aaf7a46d829180990fa26c21e3b721b6a3d223441f833e286 WatchSource:0}: Error finding container d1313b9f28ddbc2aaf7a46d829180990fa26c21e3b721b6a3d223441f833e286: Status 404 returned error can't find the container with id d1313b9f28ddbc2aaf7a46d829180990fa26c21e3b721b6a3d223441f833e286
Sep 14 08:00:50.311213 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.311164    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" event={"ID":"81e76cec-8c89-43a0-938d-7a8c74194668","Type":"ContainerStarted","Data":"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"}
Sep 14 08:00:50.311423 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.311274    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" event={"ID":"81e76cec-8c89-43a0-938d-7a8c74194668","Type":"ContainerStarted","Data":"d1313b9f28ddbc2aaf7a46d829180990fa26c21e3b721b6a3d223441f833e286"}
Sep 14 08:00:50.311423 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.311302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:00:50.312846 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.312819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" event={"ID":"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9","Type":"ContainerStarted","Data":"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"}
Sep 14 08:00:50.313074 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.313053    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:50.331933 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.331865    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" podStartSLOduration=1.33184871 podStartE2EDuration="1.33184871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:50.32922697 +0000 UTC m=+775.103725306" watchObservedRunningTime="2026-09-14 08:00:50.33184871 +0000 UTC m=+775.106347044"
Sep 14 08:00:50.348889 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:50.348842    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" podStartSLOduration=2.019654385 podStartE2EDuration="2.348830893s" totalImagesPullingTime="329.176508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:00:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:00:50.348242753 +0000 UTC m=+775.122741088" watchObservedRunningTime="2026-09-14 08:00:50.348830893 +0000 UTC m=+775.123329228"
Sep 14 08:00:52.275078 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:52.275046    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:00:55.342881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.342847    2577 generic.go:356] "Generic (PLEG): container finished" podID="76293e7e-c869-4277-812e-4c3439552980" containerID="ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4" exitCode=0
Sep 14 08:00:55.343313 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.342881    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerDied","Data":"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"}
Sep 14 08:00:55.343357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.343334    2577 scope.go:117] "RemoveContainer" containerID="ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"
Sep 14 08:00:55.344485 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.344454    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:55.344566 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.344531    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:55.344607 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:55.344566    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:00:56.322861 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:56.322832    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:00:56.350371 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:56.350341    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerStarted","Data":"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"}
Sep 14 08:00:56.929497 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:56.929460    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:00:56.929791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:56.929770    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:00:56.929835 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:56.929820    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 14 08:00:57.930482 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:57.930449    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:00:57.930876 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:57.930505    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 14 08:00:58.930038 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:58.930000    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:00:58.930215 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:58.930063    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 14 08:00:59.929688 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:59.929648    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:00:59.930133 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:00:59.929706    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 14 08:01:00.930243 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:00.930209    2577 patch_prober.go:28] interesting pod/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused" start-of-body=
Sep 14 08:01:00.930613 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:00.930265    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.44:15021/healthz/ready\": dial tcp 10.132.0.44:15021: connect: connection refused"
Sep 14 08:01:01.319809 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:01.319728    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:01:01.934149 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:01.934113    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:01:01.934633 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:01.934511    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:01:01.935509 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:01.935488    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:01:05.575828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.575801    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:01:05.578841 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.578825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.582338 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.582318    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-5ae4e6-serving-cert\""
Sep 14 08:01:05.595571 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.595540    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:01:05.703730 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.703698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.703906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.703752    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lscqs\" (UniqueName: \"kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.703906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.703894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.704060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.703955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.804560 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.804530    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.804759 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.804572    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.804759 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.804626    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.804759 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.804668    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lscqs\" (UniqueName: \"kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.804944 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:05.804768    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-serving-cert: secret "maas-api-e2e-inf-b-5ae4e6-serving-cert" not found
Sep 14 08:01:05.804944 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:05.804834    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls podName:a9fa1f9c-3996-4951-a408-a43370948282 nodeName:}" failed. No retries permitted until 2026-09-14 08:01:06.304818424 +0000 UTC m=+791.079316741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls") pod "maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" (UID: "a9fa1f9c-3996-4951-a408-a43370948282") : secret "maas-api-e2e-inf-b-5ae4e6-serving-cert" not found
Sep 14 08:01:05.805443 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.805415    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.807086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.807067    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:05.813906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:05.813888    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lscqs\" (UniqueName: \"kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:06.309685 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:06.309652    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:06.312083 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:06.312059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") pod \"maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:06.489215 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:06.489180    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:06.768081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:06.768053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:01:06.770112 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:01:06.770084    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9fa1f9c_3996_4951_a408_a43370948282.slice/crio-ae6635a720cb4b661a26e97d3b315a946b85c6adc8c2c8c2992ddb693ffeaa3d WatchSource:0}: Error finding container ae6635a720cb4b661a26e97d3b315a946b85c6adc8c2c8c2992ddb693ffeaa3d: Status 404 returned error can't find the container with id ae6635a720cb4b661a26e97d3b315a946b85c6adc8c2c8c2992ddb693ffeaa3d
Sep 14 08:01:07.416377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:07.416346    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" event={"ID":"a9fa1f9c-3996-4951-a408-a43370948282","Type":"ContainerStarted","Data":"ae6635a720cb4b661a26e97d3b315a946b85c6adc8c2c8c2992ddb693ffeaa3d"}
Sep 14 08:01:08.422792 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:08.422758    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" event={"ID":"a9fa1f9c-3996-4951-a408-a43370948282","Type":"ContainerStarted","Data":"b33331e2d8a371d0f34fda736ed4c331cf03701f924c88b73720577fda6176af"}
Sep 14 08:01:08.423201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:08.422821    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:08.445648 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:08.445603    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" podStartSLOduration=2.260026765 podStartE2EDuration="3.445587665s" totalImagesPullingTime="1.1855609s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:08.443230863 +0000 UTC m=+793.217729199" watchObservedRunningTime="2026-09-14 08:01:08.445587665 +0000 UTC m=+793.220086000"
Sep 14 08:01:14.434804 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:14.434776    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:01:14.933270 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:14.933241    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:01:14.933552 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:14.933500    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" podUID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" containerName="maas-api" containerID="cri-o://27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9" gracePeriod=30
Sep 14 08:01:15.007268 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.007234    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:01:15.007559 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.007502    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" podUID="81e76cec-8c89-43a0-938d-7a8c74194668" containerName="payload-pre-processing" containerID="cri-o://b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3" gracePeriod=30
Sep 14 08:01:15.256434 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.256411    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:01:15.283531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.283509    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:01:15.397041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397005    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghbqn\" (UniqueName: \"kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn\") pod \"81e76cec-8c89-43a0-938d-7a8c74194668\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") "
Sep 14 08:01:15.397237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397056    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs\") pod \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") "
Sep 14 08:01:15.397237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397096    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls\") pod \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") "
Sep 14 08:01:15.397237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397126    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4m29x\" (UniqueName: \"kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x\") pod \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") "
Sep 14 08:01:15.397237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume\") pod \"81e76cec-8c89-43a0-938d-7a8c74194668\" (UID: \"81e76cec-8c89-43a0-938d-7a8c74194668\") "
Sep 14 08:01:15.397237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397204    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca\") pod \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\" (UID: \"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9\") "
Sep 14 08:01:15.397967 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.397938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" (UID: "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:15.398638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.398604    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81e76cec-8c89-43a0-938d-7a8c74194668" (UID: "81e76cec-8c89-43a0-938d-7a8c74194668"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:01:15.400388 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.400360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" (UID: "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:01:15.400833 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.400810    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x" (OuterVolumeSpecName: "kube-api-access-4m29x") pod "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" (UID: "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9"). InnerVolumeSpecName "kube-api-access-4m29x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:01:15.402120 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.402096    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" (UID: "e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:01:15.402806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.402449    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn" (OuterVolumeSpecName: "kube-api-access-ghbqn") pod "81e76cec-8c89-43a0-938d-7a8c74194668" (UID: "81e76cec-8c89-43a0-938d-7a8c74194668"). InnerVolumeSpecName "kube-api-access-ghbqn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:01:15.462564 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.462530    2577 generic.go:356] "Generic (PLEG): container finished" podID="81e76cec-8c89-43a0-938d-7a8c74194668" containerID="b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3" exitCode=0
Sep 14 08:01:15.462963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.462563    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"
Sep 14 08:01:15.462963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.462613    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" event={"ID":"81e76cec-8c89-43a0-938d-7a8c74194668","Type":"ContainerDied","Data":"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"}
Sep 14 08:01:15.462963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.462671    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9" event={"ID":"81e76cec-8c89-43a0-938d-7a8c74194668","Type":"ContainerDied","Data":"d1313b9f28ddbc2aaf7a46d829180990fa26c21e3b721b6a3d223441f833e286"}
Sep 14 08:01:15.462963 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.462702    2577 scope.go:117] "RemoveContainer" containerID="b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"
Sep 14 08:01:15.464100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.464080    2577 generic.go:356] "Generic (PLEG): container finished" podID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" containerID="27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9" exitCode=0
Sep 14 08:01:15.464206 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.464110    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" event={"ID":"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9","Type":"ContainerDied","Data":"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"}
Sep 14 08:01:15.464206 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.464141    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"
Sep 14 08:01:15.464206 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.464154    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw" event={"ID":"e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9","Type":"ContainerDied","Data":"220fa09cd27025c215b7b2239cabd9c1c0998c74075d71c710c4fa10874a7c92"}
Sep 14 08:01:15.473794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.473575    2577 scope.go:117] "RemoveContainer" containerID="b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"
Sep 14 08:01:15.475328 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:15.474600    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3\": container with ID starting with b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3 not found: ID does not exist" containerID="b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"
Sep 14 08:01:15.475328 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.474638    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3"} err="failed to get container status \"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3\": rpc error: code = NotFound desc = could not find container \"b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3\": container with ID starting with b6e1821ef88314ab681ec6ac3b2e15f334437f2765917420365292eb126b5ab3 not found: ID does not exist"
Sep 14 08:01:15.475328 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.474660    2577 scope.go:117] "RemoveContainer" containerID="27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"
Sep 14 08:01:15.484888 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.484868    2577 scope.go:117] "RemoveContainer" containerID="27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"
Sep 14 08:01:15.485204 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:15.485185    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9\": container with ID starting with 27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9 not found: ID does not exist" containerID="27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"
Sep 14 08:01:15.485272 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.485211    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9"} err="failed to get container status \"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9\": rpc error: code = NotFound desc = could not find container \"27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9\": container with ID starting with 27b723dac95bd22763f2d4b42ef8b4065d8a3514e1840284890c04706f40bee9 not found: ID does not exist"
Sep 14 08:01:15.496125 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.496097    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:01:15.498421 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498402    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghbqn\" (UniqueName: \"kubernetes.io/projected/81e76cec-8c89-43a0-938d-7a8c74194668-kube-api-access-ghbqn\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.498421 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498422    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.498544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498432    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.498544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498441    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4m29x\" (UniqueName: \"kubernetes.io/projected/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-kube-api-access-4m29x\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.498544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498450    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81e76cec-8c89-43a0-938d-7a8c74194668-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.498544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.498458    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:01:15.499161 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.499142    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-a9d21f4e-84944cc54b-js8m9"]
Sep 14 08:01:15.514611 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.514587    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:01:15.522853 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.522831    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-a9d21f4e-5dc7c6b7bf-qgfsw"]
Sep 14 08:01:15.760651 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.760578    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81e76cec-8c89-43a0-938d-7a8c74194668" path="/var/lib/kubelet/pods/81e76cec-8c89-43a0-938d-7a8c74194668/volumes"
Sep 14 08:01:15.760995 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:15.760982    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" path="/var/lib/kubelet/pods/e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9/volumes"
Sep 14 08:01:21.317111 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317035    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:01:21.317475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317416    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" containerName="maas-api"
Sep 14 08:01:21.317475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317426    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" containerName="maas-api"
Sep 14 08:01:21.317475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317436    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e76cec-8c89-43a0-938d-7a8c74194668" containerName="payload-pre-processing"
Sep 14 08:01:21.317475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317441    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e76cec-8c89-43a0-938d-7a8c74194668" containerName="payload-pre-processing"
Sep 14 08:01:21.317625 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317523    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="81e76cec-8c89-43a0-938d-7a8c74194668" containerName="payload-pre-processing"
Sep 14 08:01:21.317625 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.317531    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e054d3d4-7cfb-46e0-9e9e-3b8ceaf1f0d9" containerName="maas-api"
Sep 14 08:01:21.321929 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.321899    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.325342 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.325320    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-71665efb-serving-cert\""
Sep 14 08:01:21.336250 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.336226    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:01:21.354355 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.354331    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8k8p\" (UniqueName: \"kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.354468 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.354386    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.354514 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.354466    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.354514 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.354493    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.455010 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.454974    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.455210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.455053    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.455210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.455072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.455210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.455114    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8k8p\" (UniqueName: \"kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.455210 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:21.455186    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-serving-cert: secret "maas-api-e2e-migrate-71665efb-serving-cert" not found
Sep 14 08:01:21.455491 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:01:21.455257    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls podName:5fe59382-5c62-4645-a837-138bbfe41c2f nodeName:}" failed. No retries permitted until 2026-09-14 08:01:21.955238908 +0000 UTC m=+806.729737235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls") pod "maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" (UID: "5fe59382-5c62-4645-a837-138bbfe41c2f") : secret "maas-api-e2e-migrate-71665efb-serving-cert" not found
Sep 14 08:01:21.455894 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.455867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.457689 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.457661    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.464943 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.464893    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8k8p\" (UniqueName: \"kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.959090 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.959047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:21.961464 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:21.961446    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") pod \"maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:22.198540 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.198505    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:01:22.202176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.202155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.205376 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.205352    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-71665efb-dockercfg-dqp27\""
Sep 14 08:01:22.205558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.205537    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-71665efb\""
Sep 14 08:01:22.214906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.214884    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:01:22.232508 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.232482    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:22.261179 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.261146    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.261311 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.261292    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6w6dt\" (UniqueName: \"kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.362242 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.362214    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.362608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.362357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6w6dt\" (UniqueName: \"kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.362898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.362876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.371852 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.371824    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6w6dt\" (UniqueName: \"kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt\") pod \"payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") " pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.512430 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.512407    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:01:22.514075 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.514055    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:22.515042 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:01:22.515016    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fe59382_5c62_4645_a837_138bbfe41c2f.slice/crio-27017aefd68faea15c5fce4fd03b8ba39aa277e35839b1047a95b4c24a90b639 WatchSource:0}: Error finding container 27017aefd68faea15c5fce4fd03b8ba39aa277e35839b1047a95b4c24a90b639: Status 404 returned error can't find the container with id 27017aefd68faea15c5fce4fd03b8ba39aa277e35839b1047a95b4c24a90b639
Sep 14 08:01:22.788441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:22.788406    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:01:22.789739 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:01:22.789713    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3318b66_9ce6_4bdf_9326_a072617ee1fb.slice/crio-9d89c5dd64f2c62e00d1157c596e6f2eacb6e351f3eed45e02e24f16a982f1cd WatchSource:0}: Error finding container 9d89c5dd64f2c62e00d1157c596e6f2eacb6e351f3eed45e02e24f16a982f1cd: Status 404 returned error can't find the container with id 9d89c5dd64f2c62e00d1157c596e6f2eacb6e351f3eed45e02e24f16a982f1cd
Sep 14 08:01:23.509806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.509768    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" event={"ID":"5fe59382-5c62-4645-a837-138bbfe41c2f","Type":"ContainerStarted","Data":"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"}
Sep 14 08:01:23.510277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.509834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" event={"ID":"5fe59382-5c62-4645-a837-138bbfe41c2f","Type":"ContainerStarted","Data":"27017aefd68faea15c5fce4fd03b8ba39aa277e35839b1047a95b4c24a90b639"}
Sep 14 08:01:23.510277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.510163    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:23.511682 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.511658    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" event={"ID":"b3318b66-9ce6-4bdf-9326-a072617ee1fb","Type":"ContainerStarted","Data":"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"}
Sep 14 08:01:23.511794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.511700    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" event={"ID":"b3318b66-9ce6-4bdf-9326-a072617ee1fb","Type":"ContainerStarted","Data":"9d89c5dd64f2c62e00d1157c596e6f2eacb6e351f3eed45e02e24f16a982f1cd"}
Sep 14 08:01:23.511794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.511736    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:01:23.531225 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.531182    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" podStartSLOduration=1.9634822280000002 podStartE2EDuration="2.531168431s" totalImagesPullingTime="567.686203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:23.528394897 +0000 UTC m=+808.302893232" watchObservedRunningTime="2026-09-14 08:01:23.531168431 +0000 UTC m=+808.305666769"
Sep 14 08:01:23.550647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:23.550602    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" podStartSLOduration=1.5505889910000001 podStartE2EDuration="1.550588991s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:01:23.548519716 +0000 UTC m=+808.323018054" watchObservedRunningTime="2026-09-14 08:01:23.550588991 +0000 UTC m=+808.325087325"
Sep 14 08:01:29.522866 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:29.522833    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:01:34.519331 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:01:34.519299    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:02:01.096350 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.096311    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:02:01.096859 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.096580    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" podUID="5fe59382-5c62-4645-a837-138bbfe41c2f" containerName="maas-api" containerID="cri-o://1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc" gracePeriod=30
Sep 14 08:02:01.164739 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.164706    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:02:01.165011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.164985    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" podUID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" containerName="payload-processing" containerID="cri-o://a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54" gracePeriod=30
Sep 14 08:02:01.412906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.412881    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:02:01.449063 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.449039    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:02:01.539780 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539753    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca\") pod \"5fe59382-5c62-4645-a837-138bbfe41c2f\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") "
Sep 14 08:02:01.539978 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539808    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v8k8p\" (UniqueName: \"kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p\") pod \"5fe59382-5c62-4645-a837-138bbfe41c2f\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") "
Sep 14 08:02:01.539978 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539830    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume\") pod \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") "
Sep 14 08:02:01.539978 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539884    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") pod \"5fe59382-5c62-4645-a837-138bbfe41c2f\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") "
Sep 14 08:02:01.539978 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs\") pod \"5fe59382-5c62-4645-a837-138bbfe41c2f\" (UID: \"5fe59382-5c62-4645-a837-138bbfe41c2f\") "
Sep 14 08:02:01.539978 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.539950    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6w6dt\" (UniqueName: \"kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt\") pod \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\" (UID: \"b3318b66-9ce6-4bdf-9326-a072617ee1fb\") "
Sep 14 08:02:01.540286 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.540207    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5fe59382-5c62-4645-a837-138bbfe41c2f" (UID: "5fe59382-5c62-4645-a837-138bbfe41c2f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:01.540286 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.540239    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b3318b66-9ce6-4bdf-9326-a072617ee1fb" (UID: "b3318b66-9ce6-4bdf-9326-a072617ee1fb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:01.542089 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.542062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p" (OuterVolumeSpecName: "kube-api-access-v8k8p") pod "5fe59382-5c62-4645-a837-138bbfe41c2f" (UID: "5fe59382-5c62-4645-a837-138bbfe41c2f"). InnerVolumeSpecName "kube-api-access-v8k8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:01.542089 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.542062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5fe59382-5c62-4645-a837-138bbfe41c2f" (UID: "5fe59382-5c62-4645-a837-138bbfe41c2f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:02:01.542349 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.542330    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5fe59382-5c62-4645-a837-138bbfe41c2f" (UID: "5fe59382-5c62-4645-a837-138bbfe41c2f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:02:01.542410 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.542345    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt" (OuterVolumeSpecName: "kube-api-access-6w6dt") pod "b3318b66-9ce6-4bdf-9326-a072617ee1fb" (UID: "b3318b66-9ce6-4bdf-9326-a072617ee1fb"). InnerVolumeSpecName "kube-api-access-6w6dt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641307    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641352    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5fe59382-5c62-4645-a837-138bbfe41c2f-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641366    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6w6dt\" (UniqueName: \"kubernetes.io/projected/b3318b66-9ce6-4bdf-9326-a072617ee1fb-kube-api-access-6w6dt\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641379    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5fe59382-5c62-4645-a837-138bbfe41c2f-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v8k8p\" (UniqueName: \"kubernetes.io/projected/5fe59382-5c62-4645-a837-138bbfe41c2f-kube-api-access-v8k8p\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.641412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.641421    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b3318b66-9ce6-4bdf-9326-a072617ee1fb-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:01.710366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.710327    2577 generic.go:356] "Generic (PLEG): container finished" podID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" containerID="a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54" exitCode=0
Sep 14 08:02:01.710533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.710374    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"
Sep 14 08:02:01.710533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.710411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" event={"ID":"b3318b66-9ce6-4bdf-9326-a072617ee1fb","Type":"ContainerDied","Data":"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"}
Sep 14 08:02:01.710533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.710496    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2" event={"ID":"b3318b66-9ce6-4bdf-9326-a072617ee1fb","Type":"ContainerDied","Data":"9d89c5dd64f2c62e00d1157c596e6f2eacb6e351f3eed45e02e24f16a982f1cd"}
Sep 14 08:02:01.710533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.710528    2577 scope.go:117] "RemoveContainer" containerID="a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"
Sep 14 08:02:01.711761 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.711707    2577 generic.go:356] "Generic (PLEG): container finished" podID="5fe59382-5c62-4645-a837-138bbfe41c2f" containerID="1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc" exitCode=0
Sep 14 08:02:01.711761 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.711745    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"
Sep 14 08:02:01.711761 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.711751    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" event={"ID":"5fe59382-5c62-4645-a837-138bbfe41c2f","Type":"ContainerDied","Data":"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"}
Sep 14 08:02:01.711906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.711797    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h" event={"ID":"5fe59382-5c62-4645-a837-138bbfe41c2f","Type":"ContainerDied","Data":"27017aefd68faea15c5fce4fd03b8ba39aa277e35839b1047a95b4c24a90b639"}
Sep 14 08:02:01.720401 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.720383    2577 scope.go:117] "RemoveContainer" containerID="a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"
Sep 14 08:02:01.720691 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:01.720673    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54\": container with ID starting with a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54 not found: ID does not exist" containerID="a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"
Sep 14 08:02:01.720744 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.720699    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54"} err="failed to get container status \"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54\": rpc error: code = NotFound desc = could not find container \"a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54\": container with ID starting with a8ad5a48e562a723a40e60fe6b6a2872c6673abbe13b37766eade0f871400d54 not found: ID does not exist"
Sep 14 08:02:01.720744 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.720715    2577 scope.go:117] "RemoveContainer" containerID="1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"
Sep 14 08:02:01.729272 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.729253    2577 scope.go:117] "RemoveContainer" containerID="1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"
Sep 14 08:02:01.729555 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:01.729539    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc\": container with ID starting with 1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc not found: ID does not exist" containerID="1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"
Sep 14 08:02:01.729609 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.729561    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc"} err="failed to get container status \"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc\": rpc error: code = NotFound desc = could not find container \"1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc\": container with ID starting with 1cf5a9bd367e1769a09c92dc305105be933821de208e52d19ff1ac662ac939fc not found: ID does not exist"
Sep 14 08:02:01.737243 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.737220    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:02:01.743619 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.743599    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-71665efb-7f78dc65d4-bn69h"]
Sep 14 08:02:01.754129 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.754110    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:02:01.761450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.761432    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fe59382-5c62-4645-a837-138bbfe41c2f" path="/var/lib/kubelet/pods/5fe59382-5c62-4645-a837-138bbfe41c2f/volumes"
Sep 14 08:02:01.761714 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:01.761701    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-71665efb-76b75b84c8-k6bl2"]
Sep 14 08:02:03.759877 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:03.759843    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" path="/var/lib/kubelet/pods/b3318b66-9ce6-4bdf-9326-a072617ee1fb/volumes"
Sep 14 08:02:07.818344 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818312    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:07.818699 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818690    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" containerName="payload-processing"
Sep 14 08:02:07.818743 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818701    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" containerName="payload-processing"
Sep 14 08:02:07.818743 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818738    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fe59382-5c62-4645-a837-138bbfe41c2f" containerName="maas-api"
Sep 14 08:02:07.818743 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818744    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fe59382-5c62-4645-a837-138bbfe41c2f" containerName="maas-api"
Sep 14 08:02:07.818878 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818837    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fe59382-5c62-4645-a837-138bbfe41c2f" containerName="maas-api"
Sep 14 08:02:07.818878 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.818848    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3318b66-9ce6-4bdf-9326-a072617ee1fb" containerName="payload-processing"
Sep 14 08:02:07.823445 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.823421    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:07.826387 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.826367    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-fd181c19-serving-cert\""
Sep 14 08:02:07.836895 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.836870    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:07.901058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.901016    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:07.901058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.901059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:07.901344 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.901103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf5cj\" (UniqueName: \"kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:07.901344 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:07.901202    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.002203 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.002170    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf5cj\" (UniqueName: \"kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.002377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.002227    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.002377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.002267    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.002377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.002289    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.002839 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.002819    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.004660 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.004637    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.004745 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.004685    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.012218 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.012197    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf5cj\" (UniqueName: \"kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj\") pod \"maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.134168 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.134082    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:08.414640 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.414579    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:08.416910 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:02:08.416885    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode00e7600_716f_4ee8_b833_408987dd37e3.slice/crio-adb66cd56984de71f3cbaf3f5c28e17bfd1e8ccf1c7fd134f22b8e9f8cbda356 WatchSource:0}: Error finding container adb66cd56984de71f3cbaf3f5c28e17bfd1e8ccf1c7fd134f22b8e9f8cbda356: Status 404 returned error can't find the container with id adb66cd56984de71f3cbaf3f5c28e17bfd1e8ccf1c7fd134f22b8e9f8cbda356
Sep 14 08:02:08.623057 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.623022    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:08.627726 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.627709    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.630786 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.630766    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fd181c19\""
Sep 14 08:02:08.630953 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.630936    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fd181c19-dockercfg-pdq7p\""
Sep 14 08:02:08.638073 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.638053    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:08.709320 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.709244    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.709320 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.709280    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq52f\" (UniqueName: \"kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.753870 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.753831    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" event={"ID":"e00e7600-716f-4ee8-b833-408987dd37e3","Type":"ContainerStarted","Data":"adb66cd56984de71f3cbaf3f5c28e17bfd1e8ccf1c7fd134f22b8e9f8cbda356"}
Sep 14 08:02:08.810070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.810050    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.810166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.810081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq52f\" (UniqueName: \"kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.810681 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.810664    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.819476 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.819453    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq52f\" (UniqueName: \"kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f\") pod \"payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:08.938974 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:08.938944    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:09.215287 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.215261    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:09.217263 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:02:09.217233    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82ece1e_536f_4f9f_9507_3e3b9e02c881.slice/crio-628aa881563e80a86cfa5197f0fd8934edb9d4a655dc47dcabb485c4407fd785 WatchSource:0}: Error finding container 628aa881563e80a86cfa5197f0fd8934edb9d4a655dc47dcabb485c4407fd785: Status 404 returned error can't find the container with id 628aa881563e80a86cfa5197f0fd8934edb9d4a655dc47dcabb485c4407fd785
Sep 14 08:02:09.766645 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.766620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:09.766838 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.766666    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" event={"ID":"e00e7600-716f-4ee8-b833-408987dd37e3","Type":"ContainerStarted","Data":"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"}
Sep 14 08:02:09.767757 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.767734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" event={"ID":"b82ece1e-536f-4f9f-9507-3e3b9e02c881","Type":"ContainerStarted","Data":"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"}
Sep 14 08:02:09.767874 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.767781    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" event={"ID":"b82ece1e-536f-4f9f-9507-3e3b9e02c881","Type":"ContainerStarted","Data":"628aa881563e80a86cfa5197f0fd8934edb9d4a655dc47dcabb485c4407fd785"}
Sep 14 08:02:09.767874 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.767840    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:09.788385 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.788340    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" podStartSLOduration=2.427113849 podStartE2EDuration="2.788328172s" totalImagesPullingTime="361.214323ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:09.786519844 +0000 UTC m=+854.561018179" watchObservedRunningTime="2026-09-14 08:02:09.788328172 +0000 UTC m=+854.562826506"
Sep 14 08:02:09.807358 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:09.807316    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" podStartSLOduration=1.8073027210000001 podStartE2EDuration="1.807302721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:09.806782382 +0000 UTC m=+854.581280729" watchObservedRunningTime="2026-09-14 08:02:09.807302721 +0000 UTC m=+854.581801056"
Sep 14 08:02:15.778437 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:15.778410    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:20.775300 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:20.775265    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:35.005163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.005131    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:02:35.008934 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.008907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.011856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.011833    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-814672\"/\"kube-root-ca.crt\""
Sep 14 08:02:35.011991 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.011870    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-814672\"/\"openshift-service-ca.crt\""
Sep 14 08:02:35.013110 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.013087    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-814672\"/\"default-dockercfg-kcdnz\""
Sep 14 08:02:35.013213 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.013134    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-814672\"/\"simulator-w0-814672-kserve-self-signed-certs\""
Sep 14 08:02:35.018788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.018765    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:02:35.163461 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.163634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163468    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.163634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163485    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.163634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163579    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pk4z7\" (UniqueName: \"kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.163764 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163662    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.163764 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.163689    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.264840 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.264759    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.264840 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.264800    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.264840 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.264826    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265159 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.264849    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265159 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.264904    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pk4z7\" (UniqueName: \"kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265159 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.265020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.265203    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.265220    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.265425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.265350    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.267157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.267126    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.267396 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.267377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.278107 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.278080    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pk4z7\" (UniqueName: \"kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7\") pod \"simulator-w0-814672-kserve-856468d545-h57r9\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") " pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.319639 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.319606    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:35.592533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.592508    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:02:35.594939 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:02:35.594885    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3cb5d7ea_4b1d_46ff_8f31_35bc5293a608.slice/crio-7b02f43cc6f290765f01dafe6180f119610850c00f535bdf554d835e2d594dc6 WatchSource:0}: Error finding container 7b02f43cc6f290765f01dafe6180f119610850c00f535bdf554d835e2d594dc6: Status 404 returned error can't find the container with id 7b02f43cc6f290765f01dafe6180f119610850c00f535bdf554d835e2d594dc6
Sep 14 08:02:35.927346 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.927305    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" event={"ID":"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608","Type":"ContainerStarted","Data":"fee31725d76eff15465f2ed9646dfb36588dab04547a56b5f7c216c836cfb3c0"}
Sep 14 08:02:35.927346 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.927352    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" event={"ID":"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608","Type":"ContainerStarted","Data":"7b02f43cc6f290765f01dafe6180f119610850c00f535bdf554d835e2d594dc6"}
Sep 14 08:02:35.948489 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:35.948441    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" podStartSLOduration=1.948426461 podStartE2EDuration="1.948426461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:35.947272088 +0000 UTC m=+880.721770423" watchObservedRunningTime="2026-09-14 08:02:35.948426461 +0000 UTC m=+880.722924795"
Sep 14 08:02:38.194369 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.194338    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:38.194834 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.194591    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" podUID="e00e7600-716f-4ee8-b833-408987dd37e3" containerName="maas-api" containerID="cri-o://71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f" gracePeriod=30
Sep 14 08:02:38.261532 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.261502    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:38.261732 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.261705    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" podUID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" containerName="payload-pre-processing" containerID="cri-o://ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337" gracePeriod=30
Sep 14 08:02:38.510108 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.510086    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:38.541983 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.541957    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:38.597682 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.597647    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs\") pod \"e00e7600-716f-4ee8-b833-408987dd37e3\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") "
Sep 14 08:02:38.597856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.597774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qf5cj\" (UniqueName: \"kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj\") pod \"e00e7600-716f-4ee8-b833-408987dd37e3\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") "
Sep 14 08:02:38.597856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.597838    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls\") pod \"e00e7600-716f-4ee8-b833-408987dd37e3\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") "
Sep 14 08:02:38.598022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.597978    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca\") pod \"e00e7600-716f-4ee8-b833-408987dd37e3\" (UID: \"e00e7600-716f-4ee8-b833-408987dd37e3\") "
Sep 14 08:02:38.598293 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.598270    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e00e7600-716f-4ee8-b833-408987dd37e3" (UID: "e00e7600-716f-4ee8-b833-408987dd37e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:38.598391 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.598359    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e00e7600-716f-4ee8-b833-408987dd37e3-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.599994 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.599972    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj" (OuterVolumeSpecName: "kube-api-access-qf5cj") pod "e00e7600-716f-4ee8-b833-408987dd37e3" (UID: "e00e7600-716f-4ee8-b833-408987dd37e3"). InnerVolumeSpecName "kube-api-access-qf5cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:38.600193 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.600172    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e00e7600-716f-4ee8-b833-408987dd37e3" (UID: "e00e7600-716f-4ee8-b833-408987dd37e3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:02:38.600262 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.600204    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e00e7600-716f-4ee8-b833-408987dd37e3" (UID: "e00e7600-716f-4ee8-b833-408987dd37e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:02:38.699153 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699122    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pq52f\" (UniqueName: \"kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f\") pod \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") "
Sep 14 08:02:38.699289 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699219    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume\") pod \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\" (UID: \"b82ece1e-536f-4f9f-9507-3e3b9e02c881\") "
Sep 14 08:02:38.699471 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699456    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qf5cj\" (UniqueName: \"kubernetes.io/projected/e00e7600-716f-4ee8-b833-408987dd37e3-kube-api-access-qf5cj\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.699531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699475    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.699531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699490    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e00e7600-716f-4ee8-b833-408987dd37e3-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.699594 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.699545    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b82ece1e-536f-4f9f-9507-3e3b9e02c881" (UID: "b82ece1e-536f-4f9f-9507-3e3b9e02c881"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:02:38.701011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.700990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f" (OuterVolumeSpecName: "kube-api-access-pq52f") pod "b82ece1e-536f-4f9f-9507-3e3b9e02c881" (UID: "b82ece1e-536f-4f9f-9507-3e3b9e02c881"). InnerVolumeSpecName "kube-api-access-pq52f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:02:38.800382 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.800287    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b82ece1e-536f-4f9f-9507-3e3b9e02c881-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.800382 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.800329    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pq52f\" (UniqueName: \"kubernetes.io/projected/b82ece1e-536f-4f9f-9507-3e3b9e02c881-kube-api-access-pq52f\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:02:38.943435 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.943405    2577 generic.go:356] "Generic (PLEG): container finished" podID="e00e7600-716f-4ee8-b833-408987dd37e3" containerID="71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f" exitCode=0
Sep 14 08:02:38.943573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.943443    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"
Sep 14 08:02:38.943573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.943490    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" event={"ID":"e00e7600-716f-4ee8-b833-408987dd37e3","Type":"ContainerDied","Data":"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"}
Sep 14 08:02:38.943573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.943544    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9" event={"ID":"e00e7600-716f-4ee8-b833-408987dd37e3","Type":"ContainerDied","Data":"adb66cd56984de71f3cbaf3f5c28e17bfd1e8ccf1c7fd134f22b8e9f8cbda356"}
Sep 14 08:02:38.943708 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.943575    2577 scope.go:117] "RemoveContainer" containerID="71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"
Sep 14 08:02:38.944973 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.944949    2577 generic.go:356] "Generic (PLEG): container finished" podID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" containerID="ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337" exitCode=0
Sep 14 08:02:38.945130 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.944993    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"
Sep 14 08:02:38.945130 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.945020    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" event={"ID":"b82ece1e-536f-4f9f-9507-3e3b9e02c881","Type":"ContainerDied","Data":"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"}
Sep 14 08:02:38.945130 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.945062    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls" event={"ID":"b82ece1e-536f-4f9f-9507-3e3b9e02c881","Type":"ContainerDied","Data":"628aa881563e80a86cfa5197f0fd8934edb9d4a655dc47dcabb485c4407fd785"}
Sep 14 08:02:38.952954 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.952847    2577 scope.go:117] "RemoveContainer" containerID="71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"
Sep 14 08:02:38.953246 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:38.953224    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f\": container with ID starting with 71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f not found: ID does not exist" containerID="71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"
Sep 14 08:02:38.953349 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.953258    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f"} err="failed to get container status \"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f\": rpc error: code = NotFound desc = could not find container \"71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f\": container with ID starting with 71a15df60a2d2662a8864e9eae1e0b2945513dd457ec645e32e01d4306c62f5f not found: ID does not exist"
Sep 14 08:02:38.953349 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.953278    2577 scope.go:117] "RemoveContainer" containerID="ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"
Sep 14 08:02:38.961537 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.961519    2577 scope.go:117] "RemoveContainer" containerID="ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"
Sep 14 08:02:38.961787 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:38.961770    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337\": container with ID starting with ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337 not found: ID does not exist" containerID="ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"
Sep 14 08:02:38.961835 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.961794    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337"} err="failed to get container status \"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337\": rpc error: code = NotFound desc = could not find container \"ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337\": container with ID starting with ac96acb39d377121e43a8972efa3db2f89d9184cbc1a67fac291b236a1a95337 not found: ID does not exist"
Sep 14 08:02:38.978652 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.978609    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:38.979857 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.979835    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-fd181c19-78f458ff5d-j87n9"]
Sep 14 08:02:38.994028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.994004    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:38.999627 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:38.999608    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-fd181c19-77ccbc6b6-w5pls"]
Sep 14 08:02:39.761093 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:39.761066    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" path="/var/lib/kubelet/pods/b82ece1e-536f-4f9f-9507-3e3b9e02c881/volumes"
Sep 14 08:02:39.761481 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:39.761470    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e00e7600-716f-4ee8-b833-408987dd37e3" path="/var/lib/kubelet/pods/e00e7600-716f-4ee8-b833-408987dd37e3/volumes"
Sep 14 08:02:44.326166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326136    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:02:44.326549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326526    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" containerName="payload-pre-processing"
Sep 14 08:02:44.326549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326537    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" containerName="payload-pre-processing"
Sep 14 08:02:44.326633 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326554    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e00e7600-716f-4ee8-b833-408987dd37e3" containerName="maas-api"
Sep 14 08:02:44.326633 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326560    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="e00e7600-716f-4ee8-b833-408987dd37e3" containerName="maas-api"
Sep 14 08:02:44.326633 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326626    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="e00e7600-716f-4ee8-b833-408987dd37e3" containerName="maas-api"
Sep 14 08:02:44.326734 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.326637    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b82ece1e-536f-4f9f-9507-3e3b9e02c881" containerName="payload-pre-processing"
Sep 14 08:02:44.331176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.331155    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.333910 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.333884    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b931e88a-serving-cert\""
Sep 14 08:02:44.341345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.341326    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:02:44.354298 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.354261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.354465 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.354419    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.354527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.354478    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.354527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.354515    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sszd\" (UniqueName: \"kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.456134 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.456089    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sszd\" (UniqueName: \"kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.456285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.456254    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.456405 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.456386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.456483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.456438    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.456558 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:44.456544    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-serving-cert: secret "maas-api-e2e-derive-b931e88a-serving-cert" not found
Sep 14 08:02:44.457247 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:02:44.456628    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls podName:4c4921cb-5cf7-47de-980e-4199690e2408 nodeName:}" failed. No retries permitted until 2026-09-14 08:02:44.956606232 +0000 UTC m=+889.731104560 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls") pod "maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" (UID: "4c4921cb-5cf7-47de-980e-4199690e2408") : secret "maas-api-e2e-derive-b931e88a-serving-cert" not found
Sep 14 08:02:44.457374 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.457249    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.464528 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.464508    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.467854 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.467830    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sszd\" (UniqueName: \"kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.961442 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.961410    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:44.963784 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:44.963755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") pod \"maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:45.263231 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.263134    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:45.320216 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.320188    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:45.320396 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.320383    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:45.340162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.340117    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:45.538478 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.538455    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:02:45.540983 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:02:45.540953    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4921cb_5cf7_47de_980e_4199690e2408.slice/crio-20e92368527a1b90ab6e4ddad8fd755aac7271ba8e662a92470e4b47583b136d WatchSource:0}: Error finding container 20e92368527a1b90ab6e4ddad8fd755aac7271ba8e662a92470e4b47583b136d: Status 404 returned error can't find the container with id 20e92368527a1b90ab6e4ddad8fd755aac7271ba8e662a92470e4b47583b136d
Sep 14 08:02:45.986163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.986125    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" event={"ID":"4c4921cb-5cf7-47de-980e-4199690e2408","Type":"ContainerStarted","Data":"20e92368527a1b90ab6e4ddad8fd755aac7271ba8e662a92470e4b47583b136d"}
Sep 14 08:02:45.998107 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:45.998079    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:02:46.993493 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:46.993462    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" event={"ID":"4c4921cb-5cf7-47de-980e-4199690e2408","Type":"ContainerStarted","Data":"03508d3a07c37f42261ac4e7e9f9cd6e74d262cfe28803d56c23ee5021c610f5"}
Sep 14 08:02:47.015552 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:47.015504    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" podStartSLOduration=2.539228934 podStartE2EDuration="3.01548774s" totalImagesPullingTime="476.258806ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:02:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:02:47.01301033 +0000 UTC m=+891.787508677" watchObservedRunningTime="2026-09-14 08:02:47.01548774 +0000 UTC m=+891.789986076"
Sep 14 08:02:48.003134 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:48.001655    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:54.011071 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:54.011040    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:02:55.727797 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:55.727773    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:02:55.728526 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:02:55.728510    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:03:15.922595 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:15.922561    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:03:15.923165 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:15.922775    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" podUID="4c4921cb-5cf7-47de-980e-4199690e2408" containerName="maas-api" containerID="cri-o://03508d3a07c37f42261ac4e7e9f9cd6e74d262cfe28803d56c23ee5021c610f5" gracePeriod=30
Sep 14 08:03:16.145996 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.145965    2577 generic.go:356] "Generic (PLEG): container finished" podID="4c4921cb-5cf7-47de-980e-4199690e2408" containerID="03508d3a07c37f42261ac4e7e9f9cd6e74d262cfe28803d56c23ee5021c610f5" exitCode=0
Sep 14 08:03:16.146174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.146047    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" event={"ID":"4c4921cb-5cf7-47de-980e-4199690e2408","Type":"ContainerDied","Data":"03508d3a07c37f42261ac4e7e9f9cd6e74d262cfe28803d56c23ee5021c610f5"}
Sep 14 08:03:16.174700 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.174641    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:03:16.268735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.268705    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9sszd\" (UniqueName: \"kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd\") pod \"4c4921cb-5cf7-47de-980e-4199690e2408\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") "
Sep 14 08:03:16.268893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.268748    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") pod \"4c4921cb-5cf7-47de-980e-4199690e2408\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") "
Sep 14 08:03:16.268893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.268826    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs\") pod \"4c4921cb-5cf7-47de-980e-4199690e2408\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") "
Sep 14 08:03:16.268893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.268860    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca\") pod \"4c4921cb-5cf7-47de-980e-4199690e2408\" (UID: \"4c4921cb-5cf7-47de-980e-4199690e2408\") "
Sep 14 08:03:16.269315 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.269283    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c4921cb-5cf7-47de-980e-4199690e2408" (UID: "4c4921cb-5cf7-47de-980e-4199690e2408"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:16.270953 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.270912    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c4921cb-5cf7-47de-980e-4199690e2408" (UID: "4c4921cb-5cf7-47de-980e-4199690e2408"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:16.271068 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.270966    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c4921cb-5cf7-47de-980e-4199690e2408" (UID: "4c4921cb-5cf7-47de-980e-4199690e2408"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:16.271068 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.271007    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd" (OuterVolumeSpecName: "kube-api-access-9sszd") pod "4c4921cb-5cf7-47de-980e-4199690e2408" (UID: "4c4921cb-5cf7-47de-980e-4199690e2408"). InnerVolumeSpecName "kube-api-access-9sszd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:16.370512 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.370470    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4921cb-5cf7-47de-980e-4199690e2408-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.370512 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.370506    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9sszd\" (UniqueName: \"kubernetes.io/projected/4c4921cb-5cf7-47de-980e-4199690e2408-kube-api-access-9sszd\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.370512 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.370516    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:16.370512 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:16.370524    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c4921cb-5cf7-47de-980e-4199690e2408-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:17.152593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.152568    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"
Sep 14 08:03:17.152593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.152572    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56" event={"ID":"4c4921cb-5cf7-47de-980e-4199690e2408","Type":"ContainerDied","Data":"20e92368527a1b90ab6e4ddad8fd755aac7271ba8e662a92470e4b47583b136d"}
Sep 14 08:03:17.153078 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.152636    2577 scope.go:117] "RemoveContainer" containerID="03508d3a07c37f42261ac4e7e9f9cd6e74d262cfe28803d56c23ee5021c610f5"
Sep 14 08:03:17.180985 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.180957    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:03:17.199143 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.199121    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b931e88a-69f9b75df7-c2j56"]
Sep 14 08:03:17.760608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:17.760575    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c4921cb-5cf7-47de-980e-4199690e2408" path="/var/lib/kubelet/pods/4c4921cb-5cf7-47de-980e-4199690e2408/volumes"
Sep 14 08:03:24.429579 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.429550    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:03:24.430061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.430028    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4921cb-5cf7-47de-980e-4199690e2408" containerName="maas-api"
Sep 14 08:03:24.430061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.430041    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4921cb-5cf7-47de-980e-4199690e2408" containerName="maas-api"
Sep 14 08:03:24.430154 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.430141    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c4921cb-5cf7-47de-980e-4199690e2408" containerName="maas-api"
Sep 14 08:03:24.432940 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.432907    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.437030 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.437010    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a842e\"/\"default-dockercfg-5nxbd\""
Sep 14 08:03:24.438270 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.438251    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-3a842e\"/\"test-model-3a842e-kserve-self-signed-certs\""
Sep 14 08:03:24.438384 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.438253    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a842e\"/\"kube-root-ca.crt\""
Sep 14 08:03:24.438384 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.438254    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-3a842e\"/\"openshift-service-ca.crt\""
Sep 14 08:03:24.451828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.451805    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:03:24.538217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538182    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvjmm\" (UniqueName: \"kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.538217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538225    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.538461 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538248    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.538461 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538338    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.538461 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538379    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.538461 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.538429    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639125 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639080    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639151    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639183    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639272    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639485 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639339    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvjmm\" (UniqueName: \"kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639485 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639383    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639485 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639466    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639563    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.639647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.639612    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.641402 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.641374    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.641738 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.641722    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.649271 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.649252    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvjmm\" (UniqueName: \"kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm\") pod \"test-model-3a842e-kserve-68ff87df9-rqkz7\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") " pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:24.742325 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:24.742236    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:25.030995 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:25.030961    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:03:25.032400 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:03:25.032332    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3c61766_c92e_4711_a2e6_c85ea9bb76a3.slice/crio-2ce14a44f16365c902055f57c594f922ac07676a0496ebc450e56ce0f2424128 WatchSource:0}: Error finding container 2ce14a44f16365c902055f57c594f922ac07676a0496ebc450e56ce0f2424128: Status 404 returned error can't find the container with id 2ce14a44f16365c902055f57c594f922ac07676a0496ebc450e56ce0f2424128
Sep 14 08:03:25.196242 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:25.196205    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" event={"ID":"a3c61766-c92e-4711-a2e6-c85ea9bb76a3","Type":"ContainerStarted","Data":"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"}
Sep 14 08:03:25.196242 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:25.196249    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" event={"ID":"a3c61766-c92e-4711-a2e6-c85ea9bb76a3","Type":"ContainerStarted","Data":"2ce14a44f16365c902055f57c594f922ac07676a0496ebc450e56ce0f2424128"}
Sep 14 08:03:25.230304 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:25.230187    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" podStartSLOduration=1.23016726 podStartE2EDuration="1.23016726s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:03:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:03:25.230161238 +0000 UTC m=+930.004659574" watchObservedRunningTime="2026-09-14 08:03:25.23016726 +0000 UTC m=+930.004665597"
Sep 14 08:03:34.742450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:34.742412    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:34.742889 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:34.742490    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:34.754703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:34.754678    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:35.260716 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:35.260690    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:03:53.454283 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.454247    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:03:53.454869 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.454462    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" podUID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" containerName="maas-api" containerID="cri-o://60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce" gracePeriod=30
Sep 14 08:03:53.708471 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.708412    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:03:53.822774 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.822744    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9t7d\" (UniqueName: \"kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d\") pod \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") "
Sep 14 08:03:53.822969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.822789    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs\") pod \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") "
Sep 14 08:03:53.822969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.822834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca\") pod \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") "
Sep 14 08:03:53.822969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.822857    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") pod \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\" (UID: \"ade7ae9d-8af0-454c-94ce-f4ff80d69e75\") "
Sep 14 08:03:53.823219 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.823196    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ade7ae9d-8af0-454c-94ce-f4ff80d69e75" (UID: "ade7ae9d-8af0-454c-94ce-f4ff80d69e75"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:03:53.824986 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.824955    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ade7ae9d-8af0-454c-94ce-f4ff80d69e75" (UID: "ade7ae9d-8af0-454c-94ce-f4ff80d69e75"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:53.825091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.824990    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ade7ae9d-8af0-454c-94ce-f4ff80d69e75" (UID: "ade7ae9d-8af0-454c-94ce-f4ff80d69e75"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:03:53.825091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.825043    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d" (OuterVolumeSpecName: "kube-api-access-h9t7d") pod "ade7ae9d-8af0-454c-94ce-f4ff80d69e75" (UID: "ade7ae9d-8af0-454c-94ce-f4ff80d69e75"). InnerVolumeSpecName "kube-api-access-h9t7d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:03:53.924669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.924513    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9t7d\" (UniqueName: \"kubernetes.io/projected/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-kube-api-access-h9t7d\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.924669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.924544    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.924669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.924558    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:53.924669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:53.924569    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ade7ae9d-8af0-454c-94ce-f4ff80d69e75-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:03:54.352594 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.352556    2577 generic.go:356] "Generic (PLEG): container finished" podID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" containerID="60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce" exitCode=0
Sep 14 08:03:54.352594 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.352598    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" event={"ID":"ade7ae9d-8af0-454c-94ce-f4ff80d69e75","Type":"ContainerDied","Data":"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"}
Sep 14 08:03:54.352826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.352621    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"
Sep 14 08:03:54.352826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.352661    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28" event={"ID":"ade7ae9d-8af0-454c-94ce-f4ff80d69e75","Type":"ContainerDied","Data":"8e74e7ce8dd151835526fe638085634e9bdb2f46f9fba5a3b61f9e5eb6666760"}
Sep 14 08:03:54.352826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.352682    2577 scope.go:117] "RemoveContainer" containerID="60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"
Sep 14 08:03:54.361984 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.361967    2577 scope.go:117] "RemoveContainer" containerID="60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"
Sep 14 08:03:54.362241 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:03:54.362220    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce\": container with ID starting with 60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce not found: ID does not exist" containerID="60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"
Sep 14 08:03:54.362336 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.362247    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce"} err="failed to get container status \"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce\": rpc error: code = NotFound desc = could not find container \"60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce\": container with ID starting with 60f10d316590d42ba9ba41347ed4ef5390ce2f2c078bd0d269967b4bf8753cce not found: ID does not exist"
Sep 14 08:03:54.375618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.375596    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:03:54.387499 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:54.387480    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-22b517-9f6b6568c-nfb28"]
Sep 14 08:03:55.760710 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:03:55.760677    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" path="/var/lib/kubelet/pods/ade7ae9d-8af0-454c-94ce-f4ff80d69e75/volumes"
Sep 14 08:04:00.235562 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:00.235527    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:04:06.014468 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.014435    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:04:06.015072 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.014672    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" podUID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" containerName="maas-api" containerID="cri-o://9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8" gracePeriod=30
Sep 14 08:04:06.270959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.270890    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:04:06.335353 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.335324    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca\") pod \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") "
Sep 14 08:04:06.335521 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.335364    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j2hp2\" (UniqueName: \"kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2\") pod \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") "
Sep 14 08:04:06.335521 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.335405    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls\") pod \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") "
Sep 14 08:04:06.335521 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.335427    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs\") pod \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\" (UID: \"1c563cb6-bc68-4755-9c36-c712d5e36f2d\") "
Sep 14 08:04:06.335739 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.335713    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c563cb6-bc68-4755-9c36-c712d5e36f2d" (UID: "1c563cb6-bc68-4755-9c36-c712d5e36f2d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:04:06.337569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.337538    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c563cb6-bc68-4755-9c36-c712d5e36f2d" (UID: "1c563cb6-bc68-4755-9c36-c712d5e36f2d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:06.337674 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.337573    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c563cb6-bc68-4755-9c36-c712d5e36f2d" (UID: "1c563cb6-bc68-4755-9c36-c712d5e36f2d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:06.337674 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.337614    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2" (OuterVolumeSpecName: "kube-api-access-j2hp2") pod "1c563cb6-bc68-4755-9c36-c712d5e36f2d" (UID: "1c563cb6-bc68-4755-9c36-c712d5e36f2d"). InnerVolumeSpecName "kube-api-access-j2hp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:06.414498 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.414469    2577 generic.go:356] "Generic (PLEG): container finished" podID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" containerID="9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8" exitCode=0
Sep 14 08:04:06.414673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.414518    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"
Sep 14 08:04:06.414673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.414545    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" event={"ID":"1c563cb6-bc68-4755-9c36-c712d5e36f2d","Type":"ContainerDied","Data":"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"}
Sep 14 08:04:06.414673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.414599    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg" event={"ID":"1c563cb6-bc68-4755-9c36-c712d5e36f2d","Type":"ContainerDied","Data":"f9bf2b5b3b4568a158026f086adf99acabdff9732e74f06914da69f225974e7b"}
Sep 14 08:04:06.414673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.414627    2577 scope.go:117] "RemoveContainer" containerID="9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"
Sep 14 08:04:06.425226 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.425203    2577 scope.go:117] "RemoveContainer" containerID="9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"
Sep 14 08:04:06.425504 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:04:06.425486    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8\": container with ID starting with 9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8 not found: ID does not exist" containerID="9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"
Sep 14 08:04:06.425567 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.425511    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8"} err="failed to get container status \"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8\": rpc error: code = NotFound desc = could not find container \"9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8\": container with ID starting with 9f136071f12dc4df0fb1b6bd8679ff4513ca782c1c4587e8412457e5621484f8 not found: ID does not exist"
Sep 14 08:04:06.437137 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.437105    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c563cb6-bc68-4755-9c36-c712d5e36f2d-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.437137 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.437130    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j2hp2\" (UniqueName: \"kubernetes.io/projected/1c563cb6-bc68-4755-9c36-c712d5e36f2d-kube-api-access-j2hp2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.437137 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.437140    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.437365 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.437149    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c563cb6-bc68-4755-9c36-c712d5e36f2d-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:06.439012 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.438988    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:04:06.444537 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:06.444520    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-814672-79dc87df5c-sk8hg"]
Sep 14 08:04:07.761624 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:07.761581    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" path="/var/lib/kubelet/pods/1c563cb6-bc68-4755-9c36-c712d5e36f2d/volumes"
Sep 14 08:04:15.255059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.254995    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" podUID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" containerName="main" containerID="cri-o://fee31725d76eff15465f2ed9646dfb36588dab04547a56b5f7c216c836cfb3c0" gracePeriod=45
Sep 14 08:04:15.466610 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.466578    2577 generic.go:356] "Generic (PLEG): container finished" podID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" containerID="fee31725d76eff15465f2ed9646dfb36588dab04547a56b5f7c216c836cfb3c0" exitCode=0
Sep 14 08:04:15.466789 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.466645    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" event={"ID":"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608","Type":"ContainerDied","Data":"fee31725d76eff15465f2ed9646dfb36588dab04547a56b5f7c216c836cfb3c0"}
Sep 14 08:04:15.505453 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.505384    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:04:15.624027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.623994    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pk4z7\" (UniqueName: \"kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624032    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624221 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624051    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624221 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624136    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624221 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624168    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624221 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624196    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs\") pod \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\" (UID: \"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608\") "
Sep 14 08:04:15.624401 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624278    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:15.624401 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624322    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home" (OuterVolumeSpecName: "home") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:15.624518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624438    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache" (OuterVolumeSpecName: "model-cache") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:15.624593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624557    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:15.624593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624573    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:15.624593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.624581    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:15.626321 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.626295    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7" (OuterVolumeSpecName: "kube-api-access-pk4z7") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "kube-api-access-pk4z7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:04:15.626321 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.626308    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm" (OuterVolumeSpecName: "dshm") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:04:15.626476 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.626296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" (UID: "3cb5d7ea-4b1d-46ff-8f31-35bc5293a608"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:04:15.725647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.725608    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:15.725647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.725640    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:15.725647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:15.725655    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pk4z7\" (UniqueName: \"kubernetes.io/projected/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608-kube-api-access-pk4z7\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:04:16.472744 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:16.472714    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"
Sep 14 08:04:16.473260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:16.472763    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9" event={"ID":"3cb5d7ea-4b1d-46ff-8f31-35bc5293a608","Type":"ContainerDied","Data":"7b02f43cc6f290765f01dafe6180f119610850c00f535bdf554d835e2d594dc6"}
Sep 14 08:04:16.473260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:16.472808    2577 scope.go:117] "RemoveContainer" containerID="fee31725d76eff15465f2ed9646dfb36588dab04547a56b5f7c216c836cfb3c0"
Sep 14 08:04:16.506034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:16.506006    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:04:16.507622 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:16.507600    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-814672/simulator-w0-814672-kserve-856468d545-h57r9"]
Sep 14 08:04:17.760456 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:17.760423    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" path="/var/lib/kubelet/pods/3cb5d7ea-4b1d-46ff-8f31-35bc5293a608/volumes"
Sep 14 08:04:25.411470 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.411433    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:04:25.412093 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412072    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" containerName="main"
Sep 14 08:04:25.412170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412097    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" containerName="main"
Sep 14 08:04:25.412170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412133    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" containerName="maas-api"
Sep 14 08:04:25.412170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412142    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" containerName="maas-api"
Sep 14 08:04:25.412170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412158    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" containerName="maas-api"
Sep 14 08:04:25.412170 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412167    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" containerName="maas-api"
Sep 14 08:04:25.412431 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412269    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cb5d7ea-4b1d-46ff-8f31-35bc5293a608" containerName="main"
Sep 14 08:04:25.412431 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412283    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c563cb6-bc68-4755-9c36-c712d5e36f2d" containerName="maas-api"
Sep 14 08:04:25.412431 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.412293    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ade7ae9d-8af0-454c-94ce-f4ff80d69e75" containerName="maas-api"
Sep 14 08:04:25.415777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.415754    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.420383 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.420359    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"default-dockercfg-kq58t\""
Sep 14 08:04:25.420587 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.420366    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"kube-root-ca.crt\""
Sep 14 08:04:25.420686 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.420416    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"premium-w4-b69a34-kserve-self-signed-certs\""
Sep 14 08:04:25.420686 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.420458    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"openshift-service-ca.crt\""
Sep 14 08:04:25.427703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.427682    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:04:25.516618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516585    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.516797 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516726    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.516797 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.516893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516799    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgq76\" (UniqueName: \"kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.516893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516871    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.517011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.516964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.617826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.617787    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.617860    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.617941    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.617978    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.618001    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgq76\" (UniqueName: \"kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.618034    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.618256    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.618288    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.618424 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.618407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.620161 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.620142    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.620382 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.620364    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.626977 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.626956    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgq76\" (UniqueName: \"kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76\") pod \"premium-w4-b69a34-kserve-54bb54759f-l5q6g\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") " pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:25.727233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:25.727144    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:26.005869 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:26.005844    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:04:26.007932 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:04:26.007887    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6ca66dec_ac84_4a06_bd60_616ba55963be.slice/crio-fd9c6a4f0d13f2ea406308f7ccd0765754b67776d22b38cbd988129f9bd43ba7 WatchSource:0}: Error finding container fd9c6a4f0d13f2ea406308f7ccd0765754b67776d22b38cbd988129f9bd43ba7: Status 404 returned error can't find the container with id fd9c6a4f0d13f2ea406308f7ccd0765754b67776d22b38cbd988129f9bd43ba7
Sep 14 08:04:26.526277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:26.526239    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" event={"ID":"6ca66dec-ac84-4a06-bd60-616ba55963be","Type":"ContainerStarted","Data":"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"}
Sep 14 08:04:26.526277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:26.526284    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" event={"ID":"6ca66dec-ac84-4a06-bd60-616ba55963be","Type":"ContainerStarted","Data":"fd9c6a4f0d13f2ea406308f7ccd0765754b67776d22b38cbd988129f9bd43ba7"}
Sep 14 08:04:26.548253 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:26.548205    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" podStartSLOduration=1.548188865 podStartE2EDuration="1.548188865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:26.545615058 +0000 UTC m=+991.320113392" watchObservedRunningTime="2026-09-14 08:04:26.548188865 +0000 UTC m=+991.322687199"
Sep 14 08:04:32.960354 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:32.960316    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:04:32.964830 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:32.964807    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:32.967762 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:32.967738    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-73bd4a-serving-cert\""
Sep 14 08:04:32.979662 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:32.979638    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:04:33.088991 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.088964    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.089125 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.089026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.089175 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.089147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvptb\" (UniqueName: \"kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.089208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.089179    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.190058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.190026    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvptb\" (UniqueName: \"kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.190058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.190059    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.190253 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.190189    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.190295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.190284    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.190875 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.190855    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.192676 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.192654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.192780 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.192681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.198363 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.198339    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvptb\" (UniqueName: \"kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb\") pod \"maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.282598 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.282532    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:33.556424 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.556402    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:04:33.558567 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:04:33.558538    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb6d76932_bbe3_40bf_9131_730171da8f35.slice/crio-26427abae83f36a870d1b7d852339bb4136e02158444d4e613c4fbe2ba0b774b WatchSource:0}: Error finding container 26427abae83f36a870d1b7d852339bb4136e02158444d4e613c4fbe2ba0b774b: Status 404 returned error can't find the container with id 26427abae83f36a870d1b7d852339bb4136e02158444d4e613c4fbe2ba0b774b
Sep 14 08:04:33.560399 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:33.560380    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:04:34.570812 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:34.570776    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" event={"ID":"b6d76932-bbe3-40bf-9131-730171da8f35","Type":"ContainerStarted","Data":"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"}
Sep 14 08:04:34.571194 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:34.570832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" event={"ID":"b6d76932-bbe3-40bf-9131-730171da8f35","Type":"ContainerStarted","Data":"26427abae83f36a870d1b7d852339bb4136e02158444d4e613c4fbe2ba0b774b"}
Sep 14 08:04:34.571194 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:34.570871    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:04:34.590984 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:34.590946    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" podStartSLOduration=2.163748123 podStartE2EDuration="2.590908201s" totalImagesPullingTime="427.160078ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:04:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:04:34.590091653 +0000 UTC m=+999.364589988" watchObservedRunningTime="2026-09-14 08:04:34.590908201 +0000 UTC m=+999.365406542"
Sep 14 08:04:35.728065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:35.728032    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:35.728441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:35.728086    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:35.740327 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:35.740302    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:36.591466 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:36.591439    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:04:40.581430 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:04:40.581399    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:05:08.110723 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.110687    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:08.115944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.115891    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.118344 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.118314    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-af014a32-serving-cert\""
Sep 14 08:05:08.133102 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.133077    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:08.206469 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.206439    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4bf5\" (UniqueName: \"kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.206469 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.206474    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.206680 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.206509    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.206680 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.206636    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.308192 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.308158    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.308359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.308215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4bf5\" (UniqueName: \"kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.308359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.308326    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.308439 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.308390    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.308502 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:05:08.308477    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-serving-cert: secret "maas-api-e2e-mt-af014a32-serving-cert" not found
Sep 14 08:05:08.308572 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:05:08.308560    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls podName:fdfb7c08-e510-4ae4-8161-73ef0ec99375 nodeName:}" failed. No retries permitted until 2026-09-14 08:05:08.808537302 +0000 UTC m=+1033.583035630 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls") pod "maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" (UID: "fdfb7c08-e510-4ae4-8161-73ef0ec99375") : secret "maas-api-e2e-mt-af014a32-serving-cert" not found
Sep 14 08:05:08.308894 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.308876    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.310856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.310835    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.317098 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.317074    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l4bf5\" (UniqueName: \"kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.813636 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.813588    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:08.816028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:08.815998    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") pod \"maas-api-e2e-mt-af014a32-59488b99bd-5dqdf\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:09.033619 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:09.033581    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:09.306249 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:09.306200    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:09.309316 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:05:09.309272    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdfb7c08_e510_4ae4_8161_73ef0ec99375.slice/crio-928d7206f9d79a588abda306b7cb56f67bbdde5497d4bad5088d68366ee79981 WatchSource:0}: Error finding container 928d7206f9d79a588abda306b7cb56f67bbdde5497d4bad5088d68366ee79981: Status 404 returned error can't find the container with id 928d7206f9d79a588abda306b7cb56f67bbdde5497d4bad5088d68366ee79981
Sep 14 08:05:09.749334 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:09.749301    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" event={"ID":"fdfb7c08-e510-4ae4-8161-73ef0ec99375","Type":"ContainerStarted","Data":"928d7206f9d79a588abda306b7cb56f67bbdde5497d4bad5088d68366ee79981"}
Sep 14 08:05:10.756154 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:10.756113    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" event={"ID":"fdfb7c08-e510-4ae4-8161-73ef0ec99375","Type":"ContainerStarted","Data":"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"}
Sep 14 08:05:10.756558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:10.756180    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:10.773891 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:10.773840    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" podStartSLOduration=2.335738584 podStartE2EDuration="2.773823777s" totalImagesPullingTime="438.085193ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:05:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:05:10.773000878 +0000 UTC m=+1035.547499214" watchObservedRunningTime="2026-09-14 08:05:10.773823777 +0000 UTC m=+1035.548322111"
Sep 14 08:05:16.766448 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:16.766416    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:52.690552 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.690455    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:52.691213 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.690707    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" podUID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" containerName="maas-api" containerID="cri-o://74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439" gracePeriod=30
Sep 14 08:05:52.941126 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.941068    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:52.984261 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.984235    2577 generic.go:356] "Generic (PLEG): container finished" podID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" containerID="74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439" exitCode=0
Sep 14 08:05:52.984411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.984289    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"
Sep 14 08:05:52.984411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.984295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" event={"ID":"fdfb7c08-e510-4ae4-8161-73ef0ec99375","Type":"ContainerDied","Data":"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"}
Sep 14 08:05:52.984411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.984396    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf" event={"ID":"fdfb7c08-e510-4ae4-8161-73ef0ec99375","Type":"ContainerDied","Data":"928d7206f9d79a588abda306b7cb56f67bbdde5497d4bad5088d68366ee79981"}
Sep 14 08:05:52.984533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.984418    2577 scope.go:117] "RemoveContainer" containerID="74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"
Sep 14 08:05:52.992906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.992879    2577 scope.go:117] "RemoveContainer" containerID="74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"
Sep 14 08:05:52.993225 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:05:52.993205    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439\": container with ID starting with 74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439 not found: ID does not exist" containerID="74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"
Sep 14 08:05:52.993285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:52.993235    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439"} err="failed to get container status \"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439\": rpc error: code = NotFound desc = could not find container \"74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439\": container with ID starting with 74dcc38f5aeeb1302aee81af109bc8fc3ce9b28479b2254662e0a7677980c439 not found: ID does not exist"
Sep 14 08:05:53.036554 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.036529    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs\") pod \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") "
Sep 14 08:05:53.036712 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.036588    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l4bf5\" (UniqueName: \"kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5\") pod \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") "
Sep 14 08:05:53.036712 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.036664    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca\") pod \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") "
Sep 14 08:05:53.036712 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.036691    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") pod \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\" (UID: \"fdfb7c08-e510-4ae4-8161-73ef0ec99375\") "
Sep 14 08:05:53.037079 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.037043    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fdfb7c08-e510-4ae4-8161-73ef0ec99375" (UID: "fdfb7c08-e510-4ae4-8161-73ef0ec99375"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:05:53.039076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.039051    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fdfb7c08-e510-4ae4-8161-73ef0ec99375" (UID: "fdfb7c08-e510-4ae4-8161-73ef0ec99375"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:53.039248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.039222    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fdfb7c08-e510-4ae4-8161-73ef0ec99375" (UID: "fdfb7c08-e510-4ae4-8161-73ef0ec99375"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:05:53.039248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.039237    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5" (OuterVolumeSpecName: "kube-api-access-l4bf5") pod "fdfb7c08-e510-4ae4-8161-73ef0ec99375" (UID: "fdfb7c08-e510-4ae4-8161-73ef0ec99375"). InnerVolumeSpecName "kube-api-access-l4bf5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:05:53.138294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.138259    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fdfb7c08-e510-4ae4-8161-73ef0ec99375-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.138294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.138288    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.138294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.138297    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fdfb7c08-e510-4ae4-8161-73ef0ec99375-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.138520 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.138306    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4bf5\" (UniqueName: \"kubernetes.io/projected/fdfb7c08-e510-4ae4-8161-73ef0ec99375-kube-api-access-l4bf5\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:05:53.306862 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.306825    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:53.313146 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.313117    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-af014a32-59488b99bd-5dqdf"]
Sep 14 08:05:53.760683 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:05:53.760647    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" path="/var/lib/kubelet/pods/fdfb7c08-e510-4ae4-8161-73ef0ec99375/volumes"
Sep 14 08:06:19.694799 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.694764    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:06:19.695235 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.695176    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" containerName="maas-api"
Sep 14 08:06:19.695235 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.695188    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" containerName="maas-api"
Sep 14 08:06:19.695314 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.695263    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="fdfb7c08-e510-4ae4-8161-73ef0ec99375" containerName="maas-api"
Sep 14 08:06:19.698291 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.698267    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.701114 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.701094    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-b990f4-serving-cert\""
Sep 14 08:06:19.709327 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.709299    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:06:19.792367 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.792336    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.792525 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.792378    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.792525 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.792486    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tlm57\" (UniqueName: \"kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.792600 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.792529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.893817 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.893782    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.894018 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.893850    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tlm57\" (UniqueName: \"kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.894018 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:06:19.893959    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-serving-cert: secret "maas-api-e2e-shared-a-w2-b990f4-serving-cert" not found
Sep 14 08:06:19.894124 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:06:19.894024    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls podName:ae8554b9-efdf-4b7c-9126-5d72a52ee54f nodeName:}" failed. No retries permitted until 2026-09-14 08:06:20.394007816 +0000 UTC m=+1105.168506134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls") pod "maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" (UID: "ae8554b9-efdf-4b7c-9126-5d72a52ee54f") : secret "maas-api-e2e-shared-a-w2-b990f4-serving-cert" not found
Sep 14 08:06:19.894124 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.894048    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.894294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.894141    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.894738 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.894713    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.896465 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.896447    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:19.902492 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:19.902475    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tlm57\" (UniqueName: \"kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:20.398192 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:20.398137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:20.400784 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:20.400760    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:20.610147 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:20.610111    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:20.892466 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:20.892444    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:06:20.894401 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:06:20.894375    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae8554b9_efdf_4b7c_9126_5d72a52ee54f.slice/crio-2e44bb39a0646b70f244b8c45cd905484eff1df040403e8fa4f320b198fd03a6 WatchSource:0}: Error finding container 2e44bb39a0646b70f244b8c45cd905484eff1df040403e8fa4f320b198fd03a6: Status 404 returned error can't find the container with id 2e44bb39a0646b70f244b8c45cd905484eff1df040403e8fa4f320b198fd03a6
Sep 14 08:06:21.134057 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:21.133977    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" event={"ID":"ae8554b9-efdf-4b7c-9126-5d72a52ee54f","Type":"ContainerStarted","Data":"2e44bb39a0646b70f244b8c45cd905484eff1df040403e8fa4f320b198fd03a6"}
Sep 14 08:06:22.140899 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:22.140855    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" event={"ID":"ae8554b9-efdf-4b7c-9126-5d72a52ee54f","Type":"ContainerStarted","Data":"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"}
Sep 14 08:06:22.141323 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:22.141024    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:22.178028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:22.177982    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" podStartSLOduration=2.8727895610000003 podStartE2EDuration="3.177966552s" totalImagesPullingTime="305.176991ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:22.176517103 +0000 UTC m=+1106.951015439" watchObservedRunningTime="2026-09-14 08:06:22.177966552 +0000 UTC m=+1106.952464886"
Sep 14 08:06:28.152192 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:28.152161    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:06:54.724339 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.724305    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:06:54.728254 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.728230    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.731174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.731148    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-afcf8a-serving-cert\""
Sep 14 08:06:54.740063 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.740040    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:06:54.844204 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.844168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.844204 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.844206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.844475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.844295    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nf2xc\" (UniqueName: \"kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.844475 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.844411    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.945676 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.945640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.945870 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.945707    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.945870 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.945726    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.945870 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.945755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nf2xc\" (UniqueName: \"kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.946095 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:06:54.945869    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-serving-cert: secret "maas-api-e2e-shared-b-w2-afcf8a-serving-cert" not found
Sep 14 08:06:54.946095 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:06:54.945974    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls podName:1af13d82-c326-4696-8438-af327d1a74b9 nodeName:}" failed. No retries permitted until 2026-09-14 08:06:55.445950852 +0000 UTC m=+1140.220449183 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls") pod "maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" (UID: "1af13d82-c326-4696-8438-af327d1a74b9") : secret "maas-api-e2e-shared-b-w2-afcf8a-serving-cert" not found
Sep 14 08:06:54.946402 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.946381    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.948105 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.948088    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:54.956725 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:54.956705    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nf2xc\" (UniqueName: \"kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:55.450372 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:55.450341    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:55.452827 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:55.452805    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:55.641355 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:55.641318    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:55.914962 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:55.914940    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:06:55.916694 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:06:55.916667    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1af13d82_c326_4696_8438_af327d1a74b9.slice/crio-f66400ff46a88cbdfe6cee57cfa02316958568fdc7f75a71edcab35e4caa8f5a WatchSource:0}: Error finding container f66400ff46a88cbdfe6cee57cfa02316958568fdc7f75a71edcab35e4caa8f5a: Status 404 returned error can't find the container with id f66400ff46a88cbdfe6cee57cfa02316958568fdc7f75a71edcab35e4caa8f5a
Sep 14 08:06:56.313931 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:56.313834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" event={"ID":"1af13d82-c326-4696-8438-af327d1a74b9","Type":"ContainerStarted","Data":"f66400ff46a88cbdfe6cee57cfa02316958568fdc7f75a71edcab35e4caa8f5a"}
Sep 14 08:06:57.320607 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:57.320570    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" event={"ID":"1af13d82-c326-4696-8438-af327d1a74b9","Type":"ContainerStarted","Data":"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"}
Sep 14 08:06:57.321043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:57.320642    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:06:57.339420 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:57.339361    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" podStartSLOduration=2.734012826 podStartE2EDuration="3.339342457s" totalImagesPullingTime="605.329631ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:06:57.338438501 +0000 UTC m=+1142.112936836" watchObservedRunningTime="2026-09-14 08:06:57.339342457 +0000 UTC m=+1142.113840792"
Sep 14 08:06:59.945598 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:59.945561    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:06:59.949459 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:59.949441    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:06:59.952077 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:59.952059    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-b69a34\"/\"e2e-conflict-a-cad94a07-kserve-self-signed-certs\""
Sep 14 08:06:59.958988 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:06:59.958963    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:07:00.092231 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54m2w\" (UniqueName: \"kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.092417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092265    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.092417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092289    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.092417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092324    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.092417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092348    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.092417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.092372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.193811 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.193770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.193821    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.193857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54m2w\" (UniqueName: \"kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.193906    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.193955    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.194020    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194410 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.194386    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194484 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.194407    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.194484 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.194388    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.196152 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.196087    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.196321 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.196304    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.202675 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.202657    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54m2w\" (UniqueName: \"kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w\") pod \"e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") " pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.259412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.259380    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:00.537044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:00.537018    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:07:00.538854 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:00.538827    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda1a9b39c_8d89_4b69_9f56_60c4e02b533f.slice/crio-170056cb143248c17605cde610c9b1e0e4de7eafce96b1d3947a703d3cedbb36 WatchSource:0}: Error finding container 170056cb143248c17605cde610c9b1e0e4de7eafce96b1d3947a703d3cedbb36: Status 404 returned error can't find the container with id 170056cb143248c17605cde610c9b1e0e4de7eafce96b1d3947a703d3cedbb36
Sep 14 08:07:01.350340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:01.350307    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" event={"ID":"a1a9b39c-8d89-4b69-9f56-60c4e02b533f","Type":"ContainerStarted","Data":"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"}
Sep 14 08:07:01.350340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:01.350344    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" event={"ID":"a1a9b39c-8d89-4b69-9f56-60c4e02b533f","Type":"ContainerStarted","Data":"170056cb143248c17605cde610c9b1e0e4de7eafce96b1d3947a703d3cedbb36"}
Sep 14 08:07:01.370268 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:01.370220    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" podStartSLOduration=2.370204551 podStartE2EDuration="2.370204551s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:06:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:01.368172999 +0000 UTC m=+1146.142671334" watchObservedRunningTime="2026-09-14 08:07:01.370204551 +0000 UTC m=+1146.144702885"
Sep 14 08:07:03.338841 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:03.338806    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:07:04.186805 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:04.186773    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:07:10.259835 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:10.259800    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:10.260270 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:10.259857    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:10.272568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:10.272540    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:10.415502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:10.415473    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:07:11.238904 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.238874    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-1b2b56"]
Sep 14 08:07:11.242522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.242504    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:11.245080 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.245058    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fvblv\""
Sep 14 08:07:11.249378 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.249356    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-1b2b56"]
Sep 14 08:07:11.307298 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.307268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96zx7\" (UniqueName: \"kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7\") pod \"test-curl-7861-1b2b56\" (UID: \"6d011a10-c2c4-452a-8ce2-735aafd8c860\") " pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:11.408335 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.408304    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96zx7\" (UniqueName: \"kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7\") pod \"test-curl-7861-1b2b56\" (UID: \"6d011a10-c2c4-452a-8ce2-735aafd8c860\") " pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:11.416346 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.416317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96zx7\" (UniqueName: \"kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7\") pod \"test-curl-7861-1b2b56\" (UID: \"6d011a10-c2c4-452a-8ce2-735aafd8c860\") " pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:11.554099 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.554025    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:11.829167 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:11.829140    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-1b2b56"]
Sep 14 08:07:11.831817 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:11.831789    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d011a10_c2c4_452a_8ce2_735aafd8c860.slice/crio-633f22ca4019e0919ff4e703002b1264ac0de68159090e1ba2b66fbdac8769cf WatchSource:0}: Error finding container 633f22ca4019e0919ff4e703002b1264ac0de68159090e1ba2b66fbdac8769cf: Status 404 returned error can't find the container with id 633f22ca4019e0919ff4e703002b1264ac0de68159090e1ba2b66fbdac8769cf
Sep 14 08:07:12.416828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:12.416787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-1b2b56" event={"ID":"6d011a10-c2c4-452a-8ce2-735aafd8c860","Type":"ContainerStarted","Data":"1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21"}
Sep 14 08:07:12.416828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:12.416834    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-1b2b56" event={"ID":"6d011a10-c2c4-452a-8ce2-735aafd8c860","Type":"ContainerStarted","Data":"633f22ca4019e0919ff4e703002b1264ac0de68159090e1ba2b66fbdac8769cf"}
Sep 14 08:07:12.435456 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:12.435402    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7861-1b2b56" podStartSLOduration=1.435386202 podStartE2EDuration="1.435386202s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:12.434398049 +0000 UTC m=+1157.208896384" watchObservedRunningTime="2026-09-14 08:07:12.435386202 +0000 UTC m=+1157.209884575"
Sep 14 08:07:13.035929 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.035894    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-1b2b56"]
Sep 14 08:07:13.039771 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.039747    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-1b2b56"]
Sep 14 08:07:13.151972 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.151945    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-7b2160"]
Sep 14 08:07:13.152389 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.152376    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" containerName="test-curl-7861-1b2b56"
Sep 14 08:07:13.152442 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.152391    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" containerName="test-curl-7861-1b2b56"
Sep 14 08:07:13.152484 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.152451    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" containerName="test-curl-7861-1b2b56"
Sep 14 08:07:13.155400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.155384    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:13.160496 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.160474    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-7b2160"]
Sep 14 08:07:13.225405 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.225369    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddzv8\" (UniqueName: \"kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8\") pod \"test-curl-7861-7b2160\" (UID: \"81c0ff08-f8b5-411a-aa40-2463866a6f14\") " pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:13.326310 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.326279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddzv8\" (UniqueName: \"kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8\") pod \"test-curl-7861-7b2160\" (UID: \"81c0ff08-f8b5-411a-aa40-2463866a6f14\") " pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:13.335015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.334989    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddzv8\" (UniqueName: \"kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8\") pod \"test-curl-7861-7b2160\" (UID: \"81c0ff08-f8b5-411a-aa40-2463866a6f14\") " pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:13.421801 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:13.421729    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7861-1b2b56" container="test-curl-7861-1b2b56"
Sep 14 08:07:13.466691 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.466667    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:13.735104 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:13.735075    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-7b2160"]
Sep 14 08:07:13.736693 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:13.736670    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81c0ff08_f8b5_411a_aa40_2463866a6f14.slice/crio-3e52d0a5efc95d13e66763c5d62e7f810432636451d53aba2a768e5e5fab410e WatchSource:0}: Error finding container 3e52d0a5efc95d13e66763c5d62e7f810432636451d53aba2a768e5e5fab410e: Status 404 returned error can't find the container with id 3e52d0a5efc95d13e66763c5d62e7f810432636451d53aba2a768e5e5fab410e
Sep 14 08:07:14.428763 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:14.428724    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-7b2160" event={"ID":"81c0ff08-f8b5-411a-aa40-2463866a6f14","Type":"ContainerStarted","Data":"a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63"}
Sep 14 08:07:14.429248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:14.428773    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-7b2160" event={"ID":"81c0ff08-f8b5-411a-aa40-2463866a6f14","Type":"ContainerStarted","Data":"3e52d0a5efc95d13e66763c5d62e7f810432636451d53aba2a768e5e5fab410e"}
Sep 14 08:07:14.429248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:14.428793    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7861-1b2b56" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" containerName="test-curl-7861-1b2b56" containerID="cri-o://1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21" gracePeriod=2
Sep 14 08:07:14.444753 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:14.444705    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7861-7b2160" podStartSLOduration=1.4446901859999999 podStartE2EDuration="1.444690186s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:14.442313823 +0000 UTC m=+1159.216812157" watchObservedRunningTime="2026-09-14 08:07:14.444690186 +0000 UTC m=+1159.219188575"
Sep 14 08:07:15.032658 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.032626    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-7b2160"]
Sep 14 08:07:15.036540 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.036515    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-7b2160"]
Sep 14 08:07:15.155378 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.155340    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-7ba800"]
Sep 14 08:07:15.155851 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.155831    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" containerName="test-curl-7861-7b2160"
Sep 14 08:07:15.155973 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.155854    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" containerName="test-curl-7861-7b2160"
Sep 14 08:07:15.156030 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.155980    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" containerName="test-curl-7861-7b2160"
Sep 14 08:07:15.159141 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.159122    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:15.163698 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.163675    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-7ba800"]
Sep 14 08:07:15.244010 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.243980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8zrnm\" (UniqueName: \"kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm\") pod \"test-curl-7861-7ba800\" (UID: \"aaeb8444-3028-4f14-b645-51b0c65197c6\") " pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:15.344631 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.344596    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8zrnm\" (UniqueName: \"kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm\") pod \"test-curl-7861-7ba800\" (UID: \"aaeb8444-3028-4f14-b645-51b0c65197c6\") " pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:15.353222 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.353196    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8zrnm\" (UniqueName: \"kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm\") pod \"test-curl-7861-7ba800\" (UID: \"aaeb8444-3028-4f14-b645-51b0c65197c6\") " pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:15.434243 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:15.434173    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7861-7b2160" container="test-curl-7861-7b2160"
Sep 14 08:07:15.470289 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.470266    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:15.739824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:15.739801    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-7ba800"]
Sep 14 08:07:15.741697 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:15.741670    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaaeb8444_3028_4f14_b645_51b0c65197c6.slice/crio-f1815d5fec788bd3aa9be740d33429a1d371f25e0645bd8c810b1137d728d0ed WatchSource:0}: Error finding container f1815d5fec788bd3aa9be740d33429a1d371f25e0645bd8c810b1137d728d0ed: Status 404 returned error can't find the container with id f1815d5fec788bd3aa9be740d33429a1d371f25e0645bd8c810b1137d728d0ed
Sep 14 08:07:16.442037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.442007    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-7ba800" event={"ID":"aaeb8444-3028-4f14-b645-51b0c65197c6","Type":"ContainerStarted","Data":"1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f"}
Sep 14 08:07:16.442350 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.442053    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-7ba800" event={"ID":"aaeb8444-3028-4f14-b645-51b0c65197c6","Type":"ContainerStarted","Data":"f1815d5fec788bd3aa9be740d33429a1d371f25e0645bd8c810b1137d728d0ed"}
Sep 14 08:07:16.442350 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.442171    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7861-7b2160" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" containerName="test-curl-7861-7b2160" containerID="cri-o://a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63" gracePeriod=2
Sep 14 08:07:16.457510 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.457465    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7861-7ba800" podStartSLOduration=1.457451394 podStartE2EDuration="1.457451394s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:16.456339457 +0000 UTC m=+1161.230837791" watchObservedRunningTime="2026-09-14 08:07:16.457451394 +0000 UTC m=+1161.231949729"
Sep 14 08:07:16.611336 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.611309    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:16.656990 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.656957    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96zx7\" (UniqueName: \"kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7\") pod \"6d011a10-c2c4-452a-8ce2-735aafd8c860\" (UID: \"6d011a10-c2c4-452a-8ce2-735aafd8c860\") "
Sep 14 08:07:16.659011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.658989    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7" (OuterVolumeSpecName: "kube-api-access-96zx7") pod "6d011a10-c2c4-452a-8ce2-735aafd8c860" (UID: "6d011a10-c2c4-452a-8ce2-735aafd8c860"). InnerVolumeSpecName "kube-api-access-96zx7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:16.701985 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.701202    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:07:16.701985 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.701442    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" podUID="b6d76932-bbe3-40bf-9131-730171da8f35" containerName="maas-api" containerID="cri-o://185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240" gracePeriod=30
Sep 14 08:07:16.758863 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.758830    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96zx7\" (UniqueName: \"kubernetes.io/projected/6d011a10-c2c4-452a-8ce2-735aafd8c860-kube-api-access-96zx7\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:16.945427 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.945395    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:07:16.960959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.960878    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca\") pod \"b6d76932-bbe3-40bf-9131-730171da8f35\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") "
Sep 14 08:07:16.960959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.960939    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs\") pod \"b6d76932-bbe3-40bf-9131-730171da8f35\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") "
Sep 14 08:07:16.961108 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.961002    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vvptb\" (UniqueName: \"kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb\") pod \"b6d76932-bbe3-40bf-9131-730171da8f35\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") "
Sep 14 08:07:16.961108 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.961053    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls\") pod \"b6d76932-bbe3-40bf-9131-730171da8f35\" (UID: \"b6d76932-bbe3-40bf-9131-730171da8f35\") "
Sep 14 08:07:16.961273 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.961247    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b6d76932-bbe3-40bf-9131-730171da8f35" (UID: "b6d76932-bbe3-40bf-9131-730171da8f35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:16.963240 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.963197    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b6d76932-bbe3-40bf-9131-730171da8f35" (UID: "b6d76932-bbe3-40bf-9131-730171da8f35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:16.963374 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.963351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb" (OuterVolumeSpecName: "kube-api-access-vvptb") pod "b6d76932-bbe3-40bf-9131-730171da8f35" (UID: "b6d76932-bbe3-40bf-9131-730171da8f35"). InnerVolumeSpecName "kube-api-access-vvptb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:16.963440 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:16.963367    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b6d76932-bbe3-40bf-9131-730171da8f35" (UID: "b6d76932-bbe3-40bf-9131-730171da8f35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:17.061179 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.061149    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-7ba800"]
Sep 14 08:07:17.062340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.062314    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.062340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.062340    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b6d76932-bbe3-40bf-9131-730171da8f35-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.062554 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.062354    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b6d76932-bbe3-40bf-9131-730171da8f35-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.062554 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.062368    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vvptb\" (UniqueName: \"kubernetes.io/projected/b6d76932-bbe3-40bf-9131-730171da8f35-kube-api-access-vvptb\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:17.066786 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.066760    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-7ba800"]
Sep 14 08:07:17.181559 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.181526    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7861-384919"]
Sep 14 08:07:17.181937 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.181924    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b6d76932-bbe3-40bf-9131-730171da8f35" containerName="maas-api"
Sep 14 08:07:17.181989 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.181939    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b6d76932-bbe3-40bf-9131-730171da8f35" containerName="maas-api"
Sep 14 08:07:17.181989 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.181967    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" containerName="test-curl-7861-7ba800"
Sep 14 08:07:17.181989 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.181973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" containerName="test-curl-7861-7ba800"
Sep 14 08:07:17.182081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.182032    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" containerName="test-curl-7861-7ba800"
Sep 14 08:07:17.182081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.182041    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b6d76932-bbe3-40bf-9131-730171da8f35" containerName="maas-api"
Sep 14 08:07:17.186407 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.186378    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:17.193932 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.193894    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-384919"]
Sep 14 08:07:17.264827 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.264751    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfc89\" (UniqueName: \"kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89\") pod \"test-curl-7861-384919\" (UID: \"a05a0d44-2ad3-4139-8721-46af69e85742\") " pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:17.365399 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.365365    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cfc89\" (UniqueName: \"kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89\") pod \"test-curl-7861-384919\" (UID: \"a05a0d44-2ad3-4139-8721-46af69e85742\") " pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:17.375186 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.375165    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfc89\" (UniqueName: \"kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89\") pod \"test-curl-7861-384919\" (UID: \"a05a0d44-2ad3-4139-8721-46af69e85742\") " pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:17.450684 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.450658    2577 generic.go:356] "Generic (PLEG): container finished" podID="b6d76932-bbe3-40bf-9131-730171da8f35" containerID="185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240" exitCode=0
Sep 14 08:07:17.451033 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.450713    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"
Sep 14 08:07:17.451033 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.450739    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" event={"ID":"b6d76932-bbe3-40bf-9131-730171da8f35","Type":"ContainerDied","Data":"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"}
Sep 14 08:07:17.451033 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.450787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d" event={"ID":"b6d76932-bbe3-40bf-9131-730171da8f35","Type":"ContainerDied","Data":"26427abae83f36a870d1b7d852339bb4136e02158444d4e613c4fbe2ba0b774b"}
Sep 14 08:07:17.451033 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.450817    2577 scope.go:117] "RemoveContainer" containerID="185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"
Sep 14 08:07:17.452176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.452153    2577 generic.go:356] "Generic (PLEG): container finished" podID="6d011a10-c2c4-452a-8ce2-735aafd8c860" containerID="1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21" exitCode=137
Sep 14 08:07:17.452285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.452211    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:17.452546 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:17.452528    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7861-7ba800" container="test-curl-7861-7ba800"
Sep 14 08:07:17.462002 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.461892    2577 scope.go:117] "RemoveContainer" containerID="185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"
Sep 14 08:07:17.462300 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:17.462278    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240\": container with ID starting with 185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240 not found: ID does not exist" containerID="185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"
Sep 14 08:07:17.462386 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.462305    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240"} err="failed to get container status \"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240\": rpc error: code = NotFound desc = could not find container \"185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240\": container with ID starting with 185e9ba0b260303e81fa075dae9c9fd7b73d48dea29c24d81d565d790d0fd240 not found: ID does not exist"
Sep 14 08:07:17.462386 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.462321    2577 scope.go:117] "RemoveContainer" containerID="1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21"
Sep 14 08:07:17.470091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.470076    2577 scope.go:117] "RemoveContainer" containerID="1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21"
Sep 14 08:07:17.470327 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:17.470310    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21\": container with ID starting with 1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21 not found: ID does not exist" containerID="1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21"
Sep 14 08:07:17.470387 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.470335    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21"} err="failed to get container status \"1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21\": rpc error: code = NotFound desc = could not find container \"1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21\": container with ID starting with 1196d9de11cc3a2984847f77e71d7f020bbe97fa1c1290ec7e28a3781bf71c21 not found: ID does not exist"
Sep 14 08:07:17.475570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.475547    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:07:17.481227 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.481208    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-73bd4a-55cd5f9f6b-7s65d"]
Sep 14 08:07:17.482277 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:17.482255    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-1b2b56\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" pod="opendatahub/test-curl-7861-1b2b56"
Sep 14 08:07:17.499857 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.499839    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:17.760824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.760784    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d011a10-c2c4-452a-8ce2-735aafd8c860" path="/var/lib/kubelet/pods/6d011a10-c2c4-452a-8ce2-735aafd8c860/volumes"
Sep 14 08:07:17.761296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.761278    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b6d76932-bbe3-40bf-9131-730171da8f35" path="/var/lib/kubelet/pods/b6d76932-bbe3-40bf-9131-730171da8f35/volumes"
Sep 14 08:07:17.771057 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:17.771037    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7861-384919"]
Sep 14 08:07:17.772985 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:17.772962    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda05a0d44_2ad3_4139_8721_46af69e85742.slice/crio-2b83b7bace767add7d405c49af82db6480cc216d1c7a816e002ffa1a2f7c7a72 WatchSource:0}: Error finding container 2b83b7bace767add7d405c49af82db6480cc216d1c7a816e002ffa1a2f7c7a72: Status 404 returned error can't find the container with id 2b83b7bace767add7d405c49af82db6480cc216d1c7a816e002ffa1a2f7c7a72
Sep 14 08:07:18.041523 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.041493    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:07:18.041701 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.041680    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" podUID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" containerName="maas-api" containerID="cri-o://353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc" gracePeriod=30
Sep 14 08:07:18.272638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.272615    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:07:18.376036 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.376008    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vp6z2\" (UniqueName: \"kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2\") pod \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") "
Sep 14 08:07:18.376201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.376050    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs\") pod \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") "
Sep 14 08:07:18.376201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.376075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca\") pod \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") "
Sep 14 08:07:18.376201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.376116    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") pod \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\" (UID: \"3cf6ebd1-4629-4cce-843a-4f668b8ad233\") "
Sep 14 08:07:18.376490 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.376458    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cf6ebd1-4629-4cce-843a-4f668b8ad233" (UID: "3cf6ebd1-4629-4cce-843a-4f668b8ad233"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:18.378084 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.378062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3cf6ebd1-4629-4cce-843a-4f668b8ad233" (UID: "3cf6ebd1-4629-4cce-843a-4f668b8ad233"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:18.378224 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.378203    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cf6ebd1-4629-4cce-843a-4f668b8ad233" (UID: "3cf6ebd1-4629-4cce-843a-4f668b8ad233"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:18.378278 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.378202    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2" (OuterVolumeSpecName: "kube-api-access-vp6z2") pod "3cf6ebd1-4629-4cce-843a-4f668b8ad233" (UID: "3cf6ebd1-4629-4cce-843a-4f668b8ad233"). InnerVolumeSpecName "kube-api-access-vp6z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:18.459398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.459366    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-384919" event={"ID":"a05a0d44-2ad3-4139-8721-46af69e85742","Type":"ContainerStarted","Data":"6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079"}
Sep 14 08:07:18.459722 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.459418    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7861-384919" event={"ID":"a05a0d44-2ad3-4139-8721-46af69e85742","Type":"ContainerStarted","Data":"2b83b7bace767add7d405c49af82db6480cc216d1c7a816e002ffa1a2f7c7a72"}
Sep 14 08:07:18.460744 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.460721    2577 generic.go:356] "Generic (PLEG): container finished" podID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" containerID="353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc" exitCode=0
Sep 14 08:07:18.460882 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.460754    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"
Sep 14 08:07:18.460882 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.460795    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" event={"ID":"3cf6ebd1-4629-4cce-843a-4f668b8ad233","Type":"ContainerDied","Data":"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"}
Sep 14 08:07:18.460882 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.460851    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p" event={"ID":"3cf6ebd1-4629-4cce-843a-4f668b8ad233","Type":"ContainerDied","Data":"325f9328441287a14db7283874569a37deaffc03c80aef1bb9cdc39b83a904c1"}
Sep 14 08:07:18.460882 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.460880    2577 scope.go:117] "RemoveContainer" containerID="353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"
Sep 14 08:07:18.463197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.463172    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7861-7ba800" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" containerName="test-curl-7861-7ba800" containerID="cri-o://1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f" gracePeriod=2
Sep 14 08:07:18.477015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.476987    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp6z2\" (UniqueName: \"kubernetes.io/projected/3cf6ebd1-4629-4cce-843a-4f668b8ad233-kube-api-access-vp6z2\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.477015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.477010    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.477186 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.477022    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cf6ebd1-4629-4cce-843a-4f668b8ad233-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.477186 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.477030    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cf6ebd1-4629-4cce-843a-4f668b8ad233-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:18.479891 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.479851    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7861-384919" podStartSLOduration=1.479838966 podStartE2EDuration="1.479838966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:18.478089519 +0000 UTC m=+1163.252587856" watchObservedRunningTime="2026-09-14 08:07:18.479838966 +0000 UTC m=+1163.254337301"
Sep 14 08:07:18.506397 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.506374    2577 scope.go:117] "RemoveContainer" containerID="353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"
Sep 14 08:07:18.506721 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:18.506702    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc\": container with ID starting with 353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc not found: ID does not exist" containerID="353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"
Sep 14 08:07:18.506794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.506728    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc"} err="failed to get container status \"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc\": rpc error: code = NotFound desc = could not find container \"353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc\": container with ID starting with 353ac77723a6c2c29494af933654f5251546aa0e91bd2f8cb5b5e6a23372f3cc not found: ID does not exist"
Sep 14 08:07:18.522291 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.522258    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:07:18.528013 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.527984    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-3a842e-6cd54f6cd9-4nc9p"]
Sep 14 08:07:18.591360 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.591334    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:18.678379 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.678358    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddzv8\" (UniqueName: \"kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8\") pod \"81c0ff08-f8b5-411a-aa40-2463866a6f14\" (UID: \"81c0ff08-f8b5-411a-aa40-2463866a6f14\") "
Sep 14 08:07:18.680334 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.680312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8" (OuterVolumeSpecName: "kube-api-access-ddzv8") pod "81c0ff08-f8b5-411a-aa40-2463866a6f14" (UID: "81c0ff08-f8b5-411a-aa40-2463866a6f14"). InnerVolumeSpecName "kube-api-access-ddzv8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:18.779269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:18.779206    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddzv8\" (UniqueName: \"kubernetes.io/projected/81c0ff08-f8b5-411a-aa40-2463866a6f14-kube-api-access-ddzv8\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.066182 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.066099    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7861-384919"]
Sep 14 08:07:19.070494 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.070472    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7861-384919"]
Sep 14 08:07:19.206089 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.206057    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" podUID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" containerName="main" containerID="cri-o://f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22" gracePeriod=45
Sep 14 08:07:19.461540 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.461514    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:07:19.470162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.470138    2577 generic.go:356] "Generic (PLEG): container finished" podID="81c0ff08-f8b5-411a-aa40-2463866a6f14" containerID="a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63" exitCode=137
Sep 14 08:07:19.470265 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.470172    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:19.470265 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.470235    2577 scope.go:117] "RemoveContainer" containerID="a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63"
Sep 14 08:07:19.471474 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.471454    2577 generic.go:356] "Generic (PLEG): container finished" podID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" containerID="f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22" exitCode=0
Sep 14 08:07:19.471577 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.471489    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"
Sep 14 08:07:19.471577 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.471491    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" event={"ID":"a3c61766-c92e-4711-a2e6-c85ea9bb76a3","Type":"ContainerDied","Data":"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"}
Sep 14 08:07:19.471577 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.471526    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7" event={"ID":"a3c61766-c92e-4711-a2e6-c85ea9bb76a3","Type":"ContainerDied","Data":"2ce14a44f16365c902055f57c594f922ac07676a0496ebc450e56ce0f2424128"}
Sep 14 08:07:19.471763 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:19.471750    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7861-384919" container="test-curl-7861-384919"
Sep 14 08:07:19.479501 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.479481    2577 scope.go:117] "RemoveContainer" containerID="a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63"
Sep 14 08:07:19.479839 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:19.479813    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63\": container with ID starting with a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63 not found: ID does not exist" containerID="a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63"
Sep 14 08:07:19.479949 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.479849    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63"} err="failed to get container status \"a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63\": rpc error: code = NotFound desc = could not find container \"a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63\": container with ID starting with a38031759ffa01707c90afc67e5cceb0750b46ca09fcee9644d0fae1a6077c63 not found: ID does not exist"
Sep 14 08:07:19.479949 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.479871    2577 scope.go:117] "RemoveContainer" containerID="f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"
Sep 14 08:07:19.484255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484210    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484255 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484228    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484291    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484328    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvjmm\" (UniqueName: \"kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484376    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home" (OuterVolumeSpecName: "home") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:19.484687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484498    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm\") pod \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\" (UID: \"a3c61766-c92e-4711-a2e6-c85ea9bb76a3\") "
Sep 14 08:07:19.484687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484549    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache" (OuterVolumeSpecName: "model-cache") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:19.485057 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.484760    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:19.485295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.485200    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.485295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.485223    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.485295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.485235    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.488228 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.488037    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:19.488228 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.488092    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm" (OuterVolumeSpecName: "dshm") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:07:19.488228 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:19.488174    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-7b2160\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" pod="opendatahub/test-curl-7861-7b2160"
Sep 14 08:07:19.488439 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.488275    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm" (OuterVolumeSpecName: "kube-api-access-fvjmm") pod "a3c61766-c92e-4711-a2e6-c85ea9bb76a3" (UID: "a3c61766-c92e-4711-a2e6-c85ea9bb76a3"). InnerVolumeSpecName "kube-api-access-fvjmm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:19.492814 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.492774    2577 scope.go:117] "RemoveContainer" containerID="f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"
Sep 14 08:07:19.493141 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:19.493120    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22\": container with ID starting with f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22 not found: ID does not exist" containerID="f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"
Sep 14 08:07:19.493223 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.493156    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22"} err="failed to get container status \"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22\": rpc error: code = NotFound desc = could not find container \"f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22\": container with ID starting with f3eb3084304eb4ee909e26be0fe411d1ce930140b0ec9e13b02ff4cad2793f22 not found: ID does not exist"
Sep 14 08:07:19.586620 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.586553    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.586620 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.586578    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvjmm\" (UniqueName: \"kubernetes.io/projected/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-kube-api-access-fvjmm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.586620 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.586586    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a3c61766-c92e-4711-a2e6-c85ea9bb76a3-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:19.768018 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.767989    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" path="/var/lib/kubelet/pods/3cf6ebd1-4629-4cce-843a-4f668b8ad233/volumes"
Sep 14 08:07:19.768321 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.768309    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81c0ff08-f8b5-411a-aa40-2463866a6f14" path="/var/lib/kubelet/pods/81c0ff08-f8b5-411a-aa40-2463866a6f14/volumes"
Sep 14 08:07:19.794184 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.794153    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:07:19.798268 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.798249    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-3a842e/test-model-3a842e-kserve-68ff87df9-rqkz7"]
Sep 14 08:07:19.902560 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902530    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:07:19.902959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902945    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" containerName="test-curl-7861-384919"
Sep 14 08:07:19.902959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902960    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" containerName="test-curl-7861-384919"
Sep 14 08:07:19.903065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902970    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" containerName="main"
Sep 14 08:07:19.903065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902976    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" containerName="main"
Sep 14 08:07:19.903065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902987    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" containerName="maas-api"
Sep 14 08:07:19.903065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.902992    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" containerName="maas-api"
Sep 14 08:07:19.903065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.903060    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" containerName="main"
Sep 14 08:07:19.903260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.903070    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cf6ebd1-4629-4cce-843a-4f668b8ad233" containerName="maas-api"
Sep 14 08:07:19.903260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.903077    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" containerName="test-curl-7861-384919"
Sep 14 08:07:19.907676 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.907657    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.910196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.910173    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-8b87debd-openshift-default-dockercfg-tb4p2\""
Sep 14 08:07:19.918265 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.918242    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:07:19.990668 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990795 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990795 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990886 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990795    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990886 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990886 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.990886 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.991061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990904    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q4v7j\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:19.991061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:19.990944    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092216    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092280    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092342    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092369    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092388    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092415    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092452    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q4v7j\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092479    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092508    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092622    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092809 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092678    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.092999 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.092977    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.093076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.093047    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.093126 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.093098    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.094826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.094806    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.094970 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.094951    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.100822 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.100794    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.101039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.101021    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q4v7j\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j\") pod \"e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") " pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.218912 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.218835    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:20.479386 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.479348    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7861-384919" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" containerName="test-curl-7861-384919" containerID="cri-o://6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079" gracePeriod=2
Sep 14 08:07:20.496504 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.496479    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:07:20.517785 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:20.517759    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61c54271_943e_46ee_b86e_a2e1b3d059f5.slice/crio-00e65c812c3d09d1b2757ec4e09dae6a989bb6fe4a327af0cd0c7234c09fec1d WatchSource:0}: Error finding container 00e65c812c3d09d1b2757ec4e09dae6a989bb6fe4a327af0cd0c7234c09fec1d: Status 404 returned error can't find the container with id 00e65c812c3d09d1b2757ec4e09dae6a989bb6fe4a327af0cd0c7234c09fec1d
Sep 14 08:07:20.520341 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.520301    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:20.520429 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.520373    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:20.520429 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.520409    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:20.635819 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.635791    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:20.698706 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.698678    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8zrnm\" (UniqueName: \"kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm\") pod \"aaeb8444-3028-4f14-b645-51b0c65197c6\" (UID: \"aaeb8444-3028-4f14-b645-51b0c65197c6\") "
Sep 14 08:07:20.700988 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.700958    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm" (OuterVolumeSpecName: "kube-api-access-8zrnm") pod "aaeb8444-3028-4f14-b645-51b0c65197c6" (UID: "aaeb8444-3028-4f14-b645-51b0c65197c6"). InnerVolumeSpecName "kube-api-access-8zrnm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:20.799767 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:20.799690    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8zrnm\" (UniqueName: \"kubernetes.io/projected/aaeb8444-3028-4f14-b645-51b0c65197c6-kube-api-access-8zrnm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:21.486653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.486621    2577 generic.go:356] "Generic (PLEG): container finished" podID="aaeb8444-3028-4f14-b645-51b0c65197c6" containerID="1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f" exitCode=137
Sep 14 08:07:21.487039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.486660    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:21.487039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.486718    2577 scope.go:117] "RemoveContainer" containerID="1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f"
Sep 14 08:07:21.488654 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.488619    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerStarted","Data":"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"}
Sep 14 08:07:21.488775 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.488669    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerStarted","Data":"00e65c812c3d09d1b2757ec4e09dae6a989bb6fe4a327af0cd0c7234c09fec1d"}
Sep 14 08:07:21.496366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.496351    2577 scope.go:117] "RemoveContainer" containerID="1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f"
Sep 14 08:07:21.496618 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:21.496600    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f\": container with ID starting with 1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f not found: ID does not exist" containerID="1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f"
Sep 14 08:07:21.496654 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.496626    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f"} err="failed to get container status \"1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f\": rpc error: code = NotFound desc = could not find container \"1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f\": container with ID starting with 1743724f71643730c49ba2833a1fb7ba0b909f76aecc3fd79554ce81e823244f not found: ID does not exist"
Sep 14 08:07:21.509753 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.509709    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podStartSLOduration=2.5096937820000003 podStartE2EDuration="2.509693782s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:21.509177009 +0000 UTC m=+1166.283675345" watchObservedRunningTime="2026-09-14 08:07:21.509693782 +0000 UTC m=+1166.284192119"
Sep 14 08:07:21.511415 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:21.511392    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-7ba800\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" pod="opendatahub/test-curl-7861-7ba800"
Sep 14 08:07:21.761936 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.761846    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3c61766-c92e-4711-a2e6-c85ea9bb76a3" path="/var/lib/kubelet/pods/a3c61766-c92e-4711-a2e6-c85ea9bb76a3/volumes"
Sep 14 08:07:21.762235 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:21.762222    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaeb8444-3028-4f14-b645-51b0c65197c6" path="/var/lib/kubelet/pods/aaeb8444-3028-4f14-b645-51b0c65197c6/volumes"
Sep 14 08:07:22.219201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.219169    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:22.225161 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.225138    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:22.495856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.495834    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:22.496800 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.496783    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:22.670731 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.670706    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:22.718030 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.718001    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cfc89\" (UniqueName: \"kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89\") pod \"a05a0d44-2ad3-4139-8721-46af69e85742\" (UID: \"a05a0d44-2ad3-4139-8721-46af69e85742\") "
Sep 14 08:07:22.719955 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.719933    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89" (OuterVolumeSpecName: "kube-api-access-cfc89") pod "a05a0d44-2ad3-4139-8721-46af69e85742" (UID: "a05a0d44-2ad3-4139-8721-46af69e85742"). InnerVolumeSpecName "kube-api-access-cfc89". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:22.819510 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:22.819440    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cfc89\" (UniqueName: \"kubernetes.io/projected/a05a0d44-2ad3-4139-8721-46af69e85742-kube-api-access-cfc89\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:23.502551 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.502517    2577 generic.go:356] "Generic (PLEG): container finished" podID="a05a0d44-2ad3-4139-8721-46af69e85742" containerID="6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079" exitCode=137
Sep 14 08:07:23.502997 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.502571    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:23.502997 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.502623    2577 scope.go:117] "RemoveContainer" containerID="6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079"
Sep 14 08:07:23.511877 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.511862    2577 scope.go:117] "RemoveContainer" containerID="6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079"
Sep 14 08:07:23.512180 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:23.512153    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079\": container with ID starting with 6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079 not found: ID does not exist" containerID="6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079"
Sep 14 08:07:23.512263 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.512185    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079"} err="failed to get container status \"6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079\": rpc error: code = NotFound desc = could not find container \"6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079\": container with ID starting with 6b21f393c67505273b2c379a6a12986ae0bc836fd3542a0122f17a473cd69079 not found: ID does not exist"
Sep 14 08:07:23.513469 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:23.513447    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7861-384919\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" pod="opendatahub/test-curl-7861-384919"
Sep 14 08:07:23.764519 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:23.764444    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a05a0d44-2ad3-4139-8721-46af69e85742" path="/var/lib/kubelet/pods/a05a0d44-2ad3-4139-8721-46af69e85742/volumes"
Sep 14 08:07:27.880127 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.880096    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:07:27.884259 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.884237    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:27.886835 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.886815    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-8b87debd-serving-cert\""
Sep 14 08:07:27.895270 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.895244    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:07:27.966229 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.966201    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:27.966430 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.966238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5bks\" (UniqueName: \"kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:27.966430 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.966345    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:27.966557 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:27.966427    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.067266 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.067228    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.067448 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.067299    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.067448 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.067348    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.067448 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.067371    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5bks\" (UniqueName: \"kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.067999 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.067973    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.069895 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.069873    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.069990 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.069869    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.075637 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.075604    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5bks\" (UniqueName: \"kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks\") pod \"maas-api-e2e-mt-8b87debd-7966657664-58ghq\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.196616 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.196584    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:28.469779 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.469756    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:07:28.471866 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:28.471839    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa6ed52_e066_4940_b9a0_2889e98cec0b.slice/crio-50c455b0251bf69666196cef8001d724dc323b51cd631f4253c3eb9d55771f8e WatchSource:0}: Error finding container 50c455b0251bf69666196cef8001d724dc323b51cd631f4253c3eb9d55771f8e: Status 404 returned error can't find the container with id 50c455b0251bf69666196cef8001d724dc323b51cd631f4253c3eb9d55771f8e
Sep 14 08:07:28.530015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:28.529980    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" event={"ID":"caa6ed52-e066-4940-b9a0-2889e98cec0b","Type":"ContainerStarted","Data":"50c455b0251bf69666196cef8001d724dc323b51cd631f4253c3eb9d55771f8e"}
Sep 14 08:07:29.536185 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:29.536147    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" event={"ID":"caa6ed52-e066-4940-b9a0-2889e98cec0b","Type":"ContainerStarted","Data":"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"}
Sep 14 08:07:29.536547 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:29.536210    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:29.556853 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:29.556771    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" podStartSLOduration=1.9634643999999999 podStartE2EDuration="2.556752822s" totalImagesPullingTime="593.288422ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:29.553941761 +0000 UTC m=+1174.328440090" watchObservedRunningTime="2026-09-14 08:07:29.556752822 +0000 UTC m=+1174.331251160"
Sep 14 08:07:35.546390 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:35.546359    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:07:37.426345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.426310    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:07:37.426819 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.426519    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" podUID="a9fa1f9c-3996-4951-a408-a43370948282" containerName="maas-api" containerID="cri-o://b33331e2d8a371d0f34fda736ed4c331cf03701f924c88b73720577fda6176af" gracePeriod=30
Sep 14 08:07:37.580309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.580279    2577 generic.go:356] "Generic (PLEG): container finished" podID="a9fa1f9c-3996-4951-a408-a43370948282" containerID="b33331e2d8a371d0f34fda736ed4c331cf03701f924c88b73720577fda6176af" exitCode=0
Sep 14 08:07:37.580492 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.580355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" event={"ID":"a9fa1f9c-3996-4951-a408-a43370948282","Type":"ContainerDied","Data":"b33331e2d8a371d0f34fda736ed4c331cf03701f924c88b73720577fda6176af"}
Sep 14 08:07:37.672781 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.672756    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:07:37.757368 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.757294    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lscqs\" (UniqueName: \"kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs\") pod \"a9fa1f9c-3996-4951-a408-a43370948282\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") "
Sep 14 08:07:37.757516 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.757415    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") pod \"a9fa1f9c-3996-4951-a408-a43370948282\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") "
Sep 14 08:07:37.757516 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.757449    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs\") pod \"a9fa1f9c-3996-4951-a408-a43370948282\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") "
Sep 14 08:07:37.757516 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.757466    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca\") pod \"a9fa1f9c-3996-4951-a408-a43370948282\" (UID: \"a9fa1f9c-3996-4951-a408-a43370948282\") "
Sep 14 08:07:37.757874 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.757847    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a9fa1f9c-3996-4951-a408-a43370948282" (UID: "a9fa1f9c-3996-4951-a408-a43370948282"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:07:37.759906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.759884    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a9fa1f9c-3996-4951-a408-a43370948282" (UID: "a9fa1f9c-3996-4951-a408-a43370948282"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:37.760024 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.759946    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs" (OuterVolumeSpecName: "kube-api-access-lscqs") pod "a9fa1f9c-3996-4951-a408-a43370948282" (UID: "a9fa1f9c-3996-4951-a408-a43370948282"). InnerVolumeSpecName "kube-api-access-lscqs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:07:37.760024 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.759959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a9fa1f9c-3996-4951-a408-a43370948282" (UID: "a9fa1f9c-3996-4951-a408-a43370948282"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:07:37.858299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.858265    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:37.858299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.858291    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9fa1f9c-3996-4951-a408-a43370948282-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:37.858299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.858301    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a9fa1f9c-3996-4951-a408-a43370948282-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:37.858522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:37.858310    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lscqs\" (UniqueName: \"kubernetes.io/projected/a9fa1f9c-3996-4951-a408-a43370948282-kube-api-access-lscqs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:07:38.588452 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:38.588422    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"
Sep 14 08:07:38.588452 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:38.588441    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd" event={"ID":"a9fa1f9c-3996-4951-a408-a43370948282","Type":"ContainerDied","Data":"ae6635a720cb4b661a26e97d3b315a946b85c6adc8c2c8c2992ddb693ffeaa3d"}
Sep 14 08:07:38.589013 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:38.588519    2577 scope.go:117] "RemoveContainer" containerID="b33331e2d8a371d0f34fda736ed4c331cf03701f924c88b73720577fda6176af"
Sep 14 08:07:38.608127 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:38.608099    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:07:38.612070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:38.612045    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-5ae4e6-b9f6f7ff-t42fd"]
Sep 14 08:07:39.760850 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:39.760818    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a9fa1f9c-3996-4951-a408-a43370948282" path="/var/lib/kubelet/pods/a9fa1f9c-3996-4951-a408-a43370948282/volumes"
Sep 14 08:07:48.647197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.647109    2577 generic.go:356] "Generic (PLEG): container finished" podID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerID="2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df" exitCode=0
Sep 14 08:07:48.647197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.647185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerDied","Data":"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"}
Sep 14 08:07:48.647629 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.647611    2577 scope.go:117] "RemoveContainer" containerID="2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"
Sep 14 08:07:48.648826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.648780    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:48.648891 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.648864    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:48.648944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:48.648898    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:07:49.655363 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:49.655321    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerStarted","Data":"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"}
Sep 14 08:07:50.219217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:50.219183    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:50.220131 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:50.220104    2577 patch_prober.go:28] interesting pod/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:07:50.220242 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:50.220163    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 14 08:07:51.219613 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:51.219575    2577 patch_prober.go:28] interesting pod/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:07:51.220039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:51.219641    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 14 08:07:52.219235 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.219198    2577 patch_prober.go:28] interesting pod/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:07:52.219426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.219256    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 14 08:07:52.496483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.496391    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:52.544890 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.544859    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:07:52.545283 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.545269    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a9fa1f9c-3996-4951-a408-a43370948282" containerName="maas-api"
Sep 14 08:07:52.545329 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.545285    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a9fa1f9c-3996-4951-a408-a43370948282" containerName="maas-api"
Sep 14 08:07:52.545372 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.545364    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a9fa1f9c-3996-4951-a408-a43370948282" containerName="maas-api"
Sep 14 08:07:52.548594 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.548574    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.552579 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.552560    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-7746e4-serving-cert\""
Sep 14 08:07:52.561572 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.561552    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:07:52.621641 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.621605    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.621902 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.621655    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.621902 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.621773    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.621902 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.621849    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpqdr\" (UniqueName: \"kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.723227 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.723197    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.723402 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.723251    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.723402 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.723285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpqdr\" (UniqueName: \"kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.723402 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.723344    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.723402 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:52.723391    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-serving-cert: secret "maas-api-e2e-shared-a-w3-7746e4-serving-cert" not found
Sep 14 08:07:52.723554 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:07:52.723468    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls podName:95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa nodeName:}" failed. No retries permitted until 2026-09-14 08:07:53.223447473 +0000 UTC m=+1197.997945805 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls") pod "maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" (UID: "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa") : secret "maas-api-e2e-shared-a-w3-7746e4-serving-cert" not found
Sep 14 08:07:52.723967 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.723947    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.725617 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.725601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:52.737721 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:52.737694    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpqdr\" (UniqueName: \"kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:53.220142 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.220107    2577 patch_prober.go:28] interesting pod/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:07:53.220319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.220163    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 14 08:07:53.228746 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.228710    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:53.231081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.231060    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:53.458647 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.458611    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:53.761523 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:07:53.761495    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d72e5c_d1e4_43a3_b8b2_e1e0242f1daa.slice/crio-bb7eae1d3a4c0ff8d6066df0dd3c6f59e5174e90ec9e2beda6b46da316ac68e6 WatchSource:0}: Error finding container bb7eae1d3a4c0ff8d6066df0dd3c6f59e5174e90ec9e2beda6b46da316ac68e6: Status 404 returned error can't find the container with id bb7eae1d3a4c0ff8d6066df0dd3c6f59e5174e90ec9e2beda6b46da316ac68e6
Sep 14 08:07:53.762398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:53.762376    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:07:54.219524 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.219485    2577 patch_prober.go:28] interesting pod/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body=
Sep 14 08:07:54.219696 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.219539    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused"
Sep 14 08:07:54.689105 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.689065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" event={"ID":"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa","Type":"ContainerStarted","Data":"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"}
Sep 14 08:07:54.689407 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.689387    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" event={"ID":"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa","Type":"ContainerStarted","Data":"bb7eae1d3a4c0ff8d6066df0dd3c6f59e5174e90ec9e2beda6b46da316ac68e6"}
Sep 14 08:07:54.689541 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.689528    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:07:54.709038 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:54.708984    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" podStartSLOduration=2.182301573 podStartE2EDuration="2.708969178s" totalImagesPullingTime="526.667605ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:07:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:07:54.707410247 +0000 UTC m=+1199.481908582" watchObservedRunningTime="2026-09-14 08:07:54.708969178 +0000 UTC m=+1199.483467517"
Sep 14 08:07:55.250726 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:55.250689    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:55.252119 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:55.252097    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:07:55.761681 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:55.761657    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:07:55.764335 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:07:55.764315    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:08:00.702348 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:00.702316    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:08:27.507208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.507173    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:08:27.511066 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.511044    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.513591 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.513569    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-ea9d35-serving-cert\""
Sep 14 08:08:27.522336 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.522310    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:08:27.557346 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.557314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.557474 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.557352    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.557474 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.557370    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rk589\" (UniqueName: \"kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.557474 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.557389    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.658785 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.658755    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.658968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.658793    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.658968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.658813    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rk589\" (UniqueName: \"kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.658968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.658839    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.658968 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:08:27.658931    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-serving-cert: secret "maas-api-e2e-shared-b-w3-ea9d35-serving-cert" not found
Sep 14 08:08:27.659125 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:08:27.659015    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls podName:120b1234-3f61-4655-b4f9-a42a4a22946c nodeName:}" failed. No retries permitted until 2026-09-14 08:08:28.158994459 +0000 UTC m=+1232.933492860 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls") pod "maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" (UID: "120b1234-3f61-4655-b4f9-a42a4a22946c") : secret "maas-api-e2e-shared-b-w3-ea9d35-serving-cert" not found
Sep 14 08:08:27.659532 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.659515    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.661246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.661229    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:27.669542 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:27.669519    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rk589\" (UniqueName: \"kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:28.164513 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:28.164485    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:28.167103 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:28.167070    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:28.422795 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:28.422717    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:28.694419 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:28.694389    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:08:28.695930 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:08:28.695889    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod120b1234_3f61_4655_b4f9_a42a4a22946c.slice/crio-6ccedeb5b544fbd65a6f20947ce6b999f2c4378059b84578c5cdd3cb7b2cf9de WatchSource:0}: Error finding container 6ccedeb5b544fbd65a6f20947ce6b999f2c4378059b84578c5cdd3cb7b2cf9de: Status 404 returned error can't find the container with id 6ccedeb5b544fbd65a6f20947ce6b999f2c4378059b84578c5cdd3cb7b2cf9de
Sep 14 08:08:28.875462 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:28.875424    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" event={"ID":"120b1234-3f61-4655-b4f9-a42a4a22946c","Type":"ContainerStarted","Data":"6ccedeb5b544fbd65a6f20947ce6b999f2c4378059b84578c5cdd3cb7b2cf9de"}
Sep 14 08:08:29.881653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:29.881617    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" event={"ID":"120b1234-3f61-4655-b4f9-a42a4a22946c","Type":"ContainerStarted","Data":"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"}
Sep 14 08:08:29.882040 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:29.881769    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:29.900822 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:29.900783    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" podStartSLOduration=2.442333285 podStartE2EDuration="2.900772661s" totalImagesPullingTime="458.439376ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:08:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:08:29.899358931 +0000 UTC m=+1234.673857266" watchObservedRunningTime="2026-09-14 08:08:29.900772661 +0000 UTC m=+1234.675270996"
Sep 14 08:08:35.893027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:35.892997    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:08:53.203788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:08:53.203705    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:09:08.223312 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.223275    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" podUID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" containerName="main" containerID="cri-o://d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f" gracePeriod=45
Sep 14 08:09:08.472484 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.472461    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:09:08.550569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550498    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550536    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550600    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-54m2w\" (UniqueName: \"kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550619    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550645    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir\") pod \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\" (UID: \"a1a9b39c-8d89-4b69-9f56-60c4e02b533f\") "
Sep 14 08:09:08.550824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550799    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache" (OuterVolumeSpecName: "model-cache") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:08.550824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550783    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home" (OuterVolumeSpecName: "home") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:08.551076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.550938    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:08.551076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.551033    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:08.551076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.551056    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:08.551076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.551069    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:08.552854 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.552831    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm" (OuterVolumeSpecName: "dshm") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:09:08.552965 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.552949    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w" (OuterVolumeSpecName: "kube-api-access-54m2w") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "kube-api-access-54m2w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:08.553011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.552976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a1a9b39c-8d89-4b69-9f56-60c4e02b533f" (UID: "a1a9b39c-8d89-4b69-9f56-60c4e02b533f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:08.652425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.652392    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-54m2w\" (UniqueName: \"kubernetes.io/projected/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-kube-api-access-54m2w\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:08.652425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.652420    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:08.652425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:08.652430    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a1a9b39c-8d89-4b69-9f56-60c4e02b533f-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:09.078289 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.078252    2577 generic.go:356] "Generic (PLEG): container finished" podID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" containerID="d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f" exitCode=0
Sep 14 08:09:09.078455 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.078302    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"
Sep 14 08:09:09.078455 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.078334    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" event={"ID":"a1a9b39c-8d89-4b69-9f56-60c4e02b533f","Type":"ContainerDied","Data":"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"}
Sep 14 08:09:09.078455 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.078374    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt" event={"ID":"a1a9b39c-8d89-4b69-9f56-60c4e02b533f","Type":"ContainerDied","Data":"170056cb143248c17605cde610c9b1e0e4de7eafce96b1d3947a703d3cedbb36"}
Sep 14 08:09:09.078455 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.078391    2577 scope.go:117] "RemoveContainer" containerID="d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"
Sep 14 08:09:09.091109 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.091058    2577 scope.go:117] "RemoveContainer" containerID="d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"
Sep 14 08:09:09.091378 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:09:09.091354    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f\": container with ID starting with d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f not found: ID does not exist" containerID="d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"
Sep 14 08:09:09.091481 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.091382    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f"} err="failed to get container status \"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f\": rpc error: code = NotFound desc = could not find container \"d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f\": container with ID starting with d8cce0d180778e50293fe39f42a5ed83a3136d3f0568ebea622614ad6bd8ac0f not found: ID does not exist"
Sep 14 08:09:09.103059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.103036    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:09:09.107697 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.107677    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/e2e-conflict-a-cad94a07-kserve-687844754b-fl5lt"]
Sep 14 08:09:09.759984 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:09.759954    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" path="/var/lib/kubelet/pods/a1a9b39c-8d89-4b69-9f56-60c4e02b533f/volumes"
Sep 14 08:09:11.296836 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.296804    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:09:11.297311 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.297259    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" containerName="main"
Sep 14 08:09:11.297311 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.297271    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" containerName="main"
Sep 14 08:09:11.297429 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.297340    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1a9b39c-8d89-4b69-9f56-60c4e02b533f" containerName="main"
Sep 14 08:09:11.301670 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.301649    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.305711 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.305690    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"auth-test-model-7746e4-kserve-self-signed-certs\""
Sep 14 08:09:11.305828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.305789    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"kube-root-ca.crt\""
Sep 14 08:09:11.305828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.305799    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"openshift-service-ca.crt\""
Sep 14 08:09:11.305974 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.305861    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"default-dockercfg-gtmjp\""
Sep 14 08:09:11.311284 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.311262    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:09:11.378263 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378236    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.378400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378291    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.378400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.378481 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.378522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378484    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qsdx8\" (UniqueName: \"kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.378570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.378529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.479980 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.479939    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480145 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.479995    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480145 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480057    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480145 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qsdx8\" (UniqueName: \"kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480145 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480137    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480348 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480176    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480348 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480263    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480465 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480392    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.480527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.480462    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.482303 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.482284    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.482447 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.482430    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.488334 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.488299    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qsdx8\" (UniqueName: \"kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8\") pod \"auth-test-model-7746e4-kserve-7bccc6c895-9975j\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.611626 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.611554    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:11.885781 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:11.885752    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:09:11.887895 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:09:11.887869    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod559d7414_df05_4992_8fc3_1749abc7152b.slice/crio-48d9e086841dfd869a0983cbe80c852c03ec3ea61e3e26e38d4c0a2fd694feed WatchSource:0}: Error finding container 48d9e086841dfd869a0983cbe80c852c03ec3ea61e3e26e38d4c0a2fd694feed: Status 404 returned error can't find the container with id 48d9e086841dfd869a0983cbe80c852c03ec3ea61e3e26e38d4c0a2fd694feed
Sep 14 08:09:12.097299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:12.097260    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" event={"ID":"559d7414-df05-4992-8fc3-1749abc7152b","Type":"ContainerStarted","Data":"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"}
Sep 14 08:09:12.097481 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:12.097314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" event={"ID":"559d7414-df05-4992-8fc3-1749abc7152b","Type":"ContainerStarted","Data":"48d9e086841dfd869a0983cbe80c852c03ec3ea61e3e26e38d4c0a2fd694feed"}
Sep 14 08:09:12.116294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:12.116245    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" podStartSLOduration=1.116228988 podStartE2EDuration="1.116228988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:09:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:09:12.114559101 +0000 UTC m=+1276.889057449" watchObservedRunningTime="2026-09-14 08:09:12.116228988 +0000 UTC m=+1276.890727322"
Sep 14 08:09:21.612427 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:21.612396    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:21.612427 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:21.612434    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:21.624821 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:21.624797    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:22.159056 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:22.159030    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:09:35.256691 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.256655    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:09:35.257192 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.256855    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" podUID="caa6ed52-e066-4940-b9a0-2889e98cec0b" containerName="maas-api" containerID="cri-o://37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994" gracePeriod=30
Sep 14 08:09:35.510207 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.510141    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:09:35.623579 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.623545    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca\") pod \"caa6ed52-e066-4940-b9a0-2889e98cec0b\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") "
Sep 14 08:09:35.623738 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.623625    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs\") pod \"caa6ed52-e066-4940-b9a0-2889e98cec0b\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") "
Sep 14 08:09:35.623738 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.623657    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls\") pod \"caa6ed52-e066-4940-b9a0-2889e98cec0b\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") "
Sep 14 08:09:35.623738 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.623707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5bks\" (UniqueName: \"kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks\") pod \"caa6ed52-e066-4940-b9a0-2889e98cec0b\" (UID: \"caa6ed52-e066-4940-b9a0-2889e98cec0b\") "
Sep 14 08:09:35.623933 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.623892    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "caa6ed52-e066-4940-b9a0-2889e98cec0b" (UID: "caa6ed52-e066-4940-b9a0-2889e98cec0b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:09:35.624143 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.624116    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/caa6ed52-e066-4940-b9a0-2889e98cec0b-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.625893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.625869    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks" (OuterVolumeSpecName: "kube-api-access-b5bks") pod "caa6ed52-e066-4940-b9a0-2889e98cec0b" (UID: "caa6ed52-e066-4940-b9a0-2889e98cec0b"). InnerVolumeSpecName "kube-api-access-b5bks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:09:35.625893 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.625883    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "caa6ed52-e066-4940-b9a0-2889e98cec0b" (UID: "caa6ed52-e066-4940-b9a0-2889e98cec0b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:35.626071 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.625910    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "caa6ed52-e066-4940-b9a0-2889e98cec0b" (UID: "caa6ed52-e066-4940-b9a0-2889e98cec0b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:09:35.725531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.725486    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.725531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.725526    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/caa6ed52-e066-4940-b9a0-2889e98cec0b-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:35.725531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:35.725536    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5bks\" (UniqueName: \"kubernetes.io/projected/caa6ed52-e066-4940-b9a0-2889e98cec0b-kube-api-access-b5bks\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:09:36.223091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.223061    2577 generic.go:356] "Generic (PLEG): container finished" podID="caa6ed52-e066-4940-b9a0-2889e98cec0b" containerID="37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994" exitCode=0
Sep 14 08:09:36.223296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.223126    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"
Sep 14 08:09:36.223296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.223152    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" event={"ID":"caa6ed52-e066-4940-b9a0-2889e98cec0b","Type":"ContainerDied","Data":"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"}
Sep 14 08:09:36.223296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.223216    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq" event={"ID":"caa6ed52-e066-4940-b9a0-2889e98cec0b","Type":"ContainerDied","Data":"50c455b0251bf69666196cef8001d724dc323b51cd631f4253c3eb9d55771f8e"}
Sep 14 08:09:36.223296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.223244    2577 scope.go:117] "RemoveContainer" containerID="37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"
Sep 14 08:09:36.232359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.232342    2577 scope.go:117] "RemoveContainer" containerID="37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"
Sep 14 08:09:36.232616 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:09:36.232595    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994\": container with ID starting with 37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994 not found: ID does not exist" containerID="37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"
Sep 14 08:09:36.232711 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.232622    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994"} err="failed to get container status \"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994\": rpc error: code = NotFound desc = could not find container \"37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994\": container with ID starting with 37394b44caa21b9a179d98e80c84c5eb987ea48c5c02f28eea736875a7cb6994 not found: ID does not exist"
Sep 14 08:09:36.240533 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.240499    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:09:36.244192 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:36.244171    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-8b87debd-7966657664-58ghq"]
Sep 14 08:09:37.760969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:09:37.760939    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="caa6ed52-e066-4940-b9a0-2889e98cec0b" path="/var/lib/kubelet/pods/caa6ed52-e066-4940-b9a0-2889e98cec0b/volumes"
Sep 14 08:10:26.846932 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:26.846833    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:10:26.847478 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:26.847164    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy" containerID="cri-o://34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840" gracePeriod=30
Sep 14 08:10:32.103834 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.103808    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:10:32.179842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.179809    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.179876    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q4v7j\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.179908    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.179955    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180014    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180212 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180066    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180212 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180128    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180212 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180157    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180212 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180162    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:32.180212 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180202    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token\") pod \"61c54271-943e-46ee-b86e-a2e1b3d059f5\" (UID: \"61c54271-943e-46ee-b86e-a2e1b3d059f5\") "
Sep 14 08:10:32.180479 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180298    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:32.180479 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180360    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:32.180479 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180370    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data" (OuterVolumeSpecName: "istio-data") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:32.180635 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180561    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:10:32.180701 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180631    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.180701 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180653    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.180701 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180667    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.180701 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.180679    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.182336 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.182306    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j" (OuterVolumeSpecName: "kube-api-access-q4v7j") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "kube-api-access-q4v7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:10:32.182473 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.182336    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:10:32.182473 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.182382    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:10:32.182473 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.182465    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token" (OuterVolumeSpecName: "istio-token") pod "61c54271-943e-46ee-b86e-a2e1b3d059f5" (UID: "61c54271-943e-46ee-b86e-a2e1b3d059f5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:10:32.282195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.282163    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.282195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.282194    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/61c54271-943e-46ee-b86e-a2e1b3d059f5-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.282405 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.282205    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.282405 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.282218    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q4v7j\" (UniqueName: \"kubernetes.io/projected/61c54271-943e-46ee-b86e-a2e1b3d059f5-kube-api-access-q4v7j\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.282405 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.282240    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/61c54271-943e-46ee-b86e-a2e1b3d059f5-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:10:32.514776 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.514680    2577 generic.go:356] "Generic (PLEG): container finished" podID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerID="34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840" exitCode=0
Sep 14 08:10:32.514776 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.514733    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"
Sep 14 08:10:32.514776 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.514764    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerDied","Data":"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"}
Sep 14 08:10:32.515050 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.514819    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg" event={"ID":"61c54271-943e-46ee-b86e-a2e1b3d059f5","Type":"ContainerDied","Data":"00e65c812c3d09d1b2757ec4e09dae6a989bb6fe4a327af0cd0c7234c09fec1d"}
Sep 14 08:10:32.515050 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.514841    2577 scope.go:117] "RemoveContainer" containerID="34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"
Sep 14 08:10:32.524312 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.524294    2577 scope.go:117] "RemoveContainer" containerID="2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"
Sep 14 08:10:32.532516 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.532496    2577 scope.go:117] "RemoveContainer" containerID="34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"
Sep 14 08:10:32.532764 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:10:32.532745    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840\": container with ID starting with 34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840 not found: ID does not exist" containerID="34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"
Sep 14 08:10:32.532816 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.532773    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840"} err="failed to get container status \"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840\": rpc error: code = NotFound desc = could not find container \"34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840\": container with ID starting with 34a3ac54e40be68c0649529cf42f52a25dd800f3363d908ed6304540b2ac0840 not found: ID does not exist"
Sep 14 08:10:32.532816 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.532790    2577 scope.go:117] "RemoveContainer" containerID="2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"
Sep 14 08:10:32.533063 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:10:32.533043    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df\": container with ID starting with 2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df not found: ID does not exist" containerID="2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"
Sep 14 08:10:32.533129 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.533071    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df"} err="failed to get container status \"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df\": rpc error: code = NotFound desc = could not find container \"2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df\": container with ID starting with 2329b5bc402881a5438b1cb3e6722bb4ae38cfe993d7ad7a5ff728c2717a92df not found: ID does not exist"
Sep 14 08:10:32.538631 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.538610    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:10:32.541821 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:32.541795    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-8b87debd-openshift-default-d5597fc84-wcwvg"]
Sep 14 08:10:33.760958 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:10:33.760902    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" path="/var/lib/kubelet/pods/61c54271-943e-46ee-b86e-a2e1b3d059f5/volumes"
Sep 14 08:11:28.319748 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.319713    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:11:28.320388 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320367    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:28.320467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320391    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:28.320467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320404    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="caa6ed52-e066-4940-b9a0-2889e98cec0b" containerName="maas-api"
Sep 14 08:11:28.320467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320411    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="caa6ed52-e066-4940-b9a0-2889e98cec0b" containerName="maas-api"
Sep 14 08:11:28.320467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320437    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:28.320467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:28.320724 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320563    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:28.320724 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.320579    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="caa6ed52-e066-4940-b9a0-2889e98cec0b" containerName="maas-api"
Sep 14 08:11:28.323650 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.323630    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.326420 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.326401    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-b914f7-serving-cert\""
Sep 14 08:11:28.334845 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.334817    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:11:28.426256 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.426223    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skp25\" (UniqueName: \"kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.426256 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.426258    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.426483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.426372    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.426483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.426408    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.528004 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.527973    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.528174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.528014    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.528174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.528072    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skp25\" (UniqueName: \"kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.528174 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.528093    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.528347 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:11:28.528225    2577 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-serving-cert: secret "maas-api-e2e-api-a-b914f7-serving-cert" not found
Sep 14 08:11:28.528347 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:11:28.528305    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls podName:b0d9a99a-902f-4fa4-a1b6-a58e3f848d21 nodeName:}" failed. No retries permitted until 2026-09-14 08:11:29.028285159 +0000 UTC m=+1413.802783488 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls") pod "maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" (UID: "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21") : secret "maas-api-e2e-api-a-b914f7-serving-cert" not found
Sep 14 08:11:28.528601 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.528581    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.530584 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.530562    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:28.539698 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:28.539676    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-skp25\" (UniqueName: \"kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:29.034082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.034047    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:29.036454 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.036433    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") pod \"maas-api-e2e-api-a-b914f7-6b99474756-9pcgc\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:29.235975 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.235911    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:29.506473 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.506448    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:11:29.507953 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:11:29.507926    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0d9a99a_902f_4fa4_a1b6_a58e3f848d21.slice/crio-15b9b00cef952b44e1f03171125d2afa26ea171ab93c4abdc718e0e00b6b330f WatchSource:0}: Error finding container 15b9b00cef952b44e1f03171125d2afa26ea171ab93c4abdc718e0e00b6b330f: Status 404 returned error can't find the container with id 15b9b00cef952b44e1f03171125d2afa26ea171ab93c4abdc718e0e00b6b330f
Sep 14 08:11:29.509257 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.509241    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:11:29.801424 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:29.801380    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" event={"ID":"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21","Type":"ContainerStarted","Data":"15b9b00cef952b44e1f03171125d2afa26ea171ab93c4abdc718e0e00b6b330f"}
Sep 14 08:11:30.808459 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:30.808429    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" event={"ID":"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21","Type":"ContainerStarted","Data":"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"}
Sep 14 08:11:30.808842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:30.808519    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:30.839679 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:30.839628    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" podStartSLOduration=2.40640666 podStartE2EDuration="2.83961353s" totalImagesPullingTime="433.20687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:30.838394817 +0000 UTC m=+1415.612893151" watchObservedRunningTime="2026-09-14 08:11:30.83961353 +0000 UTC m=+1415.614111865"
Sep 14 08:11:36.819076 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:36.819047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:11:55.005023 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.004932    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:11:55.005484 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.005439    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="61c54271-943e-46ee-b86e-a2e1b3d059f5" containerName="istio-proxy"
Sep 14 08:11:55.008765 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.008747    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.011774 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.011755    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-87b7cb-openshift-default-dockercfg-8n8dn\""
Sep 14 08:11:55.018952 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.018905    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:11:55.082360 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082316    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082360 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082364    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmz58\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082431    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082462    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082505    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082555    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082831 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082640    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.082831 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.082718    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183332    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183373    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183409    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmz58\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183431    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183502 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183484    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183525    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183755    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.183898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183852    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.184084 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.183958    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.184084 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.184003    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.184218 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.184195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.185850 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.185831    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.186008 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.185991    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.191595 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.191567    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.191683 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.191568    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmz58\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58\") pod \"e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") " pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.319624 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.319548    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:55.597645 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.597616    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:11:55.600845 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:11:55.600798    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff6544eb_c57a_41e1_921c_7327168d38fd.slice/crio-9617c2e104403fe085395b157b59b6c7e04c78d21cc5fc942149e77e8e54c042 WatchSource:0}: Error finding container 9617c2e104403fe085395b157b59b6c7e04c78d21cc5fc942149e77e8e54c042: Status 404 returned error can't find the container with id 9617c2e104403fe085395b157b59b6c7e04c78d21cc5fc942149e77e8e54c042
Sep 14 08:11:55.603569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.603534    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:11:55.603688 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.603615    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:11:55.603688 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.603667    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:11:55.933033 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.932996    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerStarted","Data":"a11da3e0aa04d830f743b3f10c246fa45456bc8965280ea398c16866e04a7208"}
Sep 14 08:11:55.933196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.933048    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerStarted","Data":"9617c2e104403fe085395b157b59b6c7e04c78d21cc5fc942149e77e8e54c042"}
Sep 14 08:11:55.953067 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:55.953018    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podStartSLOduration=1.953002288 podStartE2EDuration="1.953002288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:11:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:11:55.951860478 +0000 UTC m=+1440.726358813" watchObservedRunningTime="2026-09-14 08:11:55.953002288 +0000 UTC m=+1440.727500623"
Sep 14 08:11:56.320123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:56.320047    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:56.325383 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:56.325361    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:56.937619 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:56.937591    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:11:56.938415 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:11:56.938397    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:12:03.550354 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.550321    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:12:03.554100 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.554084    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.556558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.556543    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-87b7cb-dockercfg-vhfv6\""
Sep 14 08:12:03.556900 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.556883    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-87b7cb\""
Sep 14 08:12:03.563667 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.563646    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:12:03.673095 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.673056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxpp7\" (UniqueName: \"kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.673269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.673133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.774460 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.774430    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxpp7\" (UniqueName: \"kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.774623 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.774470    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.775132 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.775112    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.783344 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.783317    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxpp7\" (UniqueName: \"kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7\") pod \"payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:03.865673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:03.865600    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:04.136569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:04.136546    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:12:04.138963 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:12:04.138934    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd2960c4f_ac58_4145_9ef4_293864290833.slice/crio-eda8b5110e973a28a8fc808b0d83c3d43fc6b777beaf383a55fb722c1a2a507d WatchSource:0}: Error finding container eda8b5110e973a28a8fc808b0d83c3d43fc6b777beaf383a55fb722c1a2a507d: Status 404 returned error can't find the container with id eda8b5110e973a28a8fc808b0d83c3d43fc6b777beaf383a55fb722c1a2a507d
Sep 14 08:12:04.984144 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:04.984088    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" event={"ID":"d2960c4f-ac58-4145-9ef4-293864290833","Type":"ContainerStarted","Data":"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"}
Sep 14 08:12:04.984629 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:04.984178    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:04.984629 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:04.984206    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" event={"ID":"d2960c4f-ac58-4145-9ef4-293864290833","Type":"ContainerStarted","Data":"eda8b5110e973a28a8fc808b0d83c3d43fc6b777beaf383a55fb722c1a2a507d"}
Sep 14 08:12:05.002249 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:05.002202    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" podStartSLOduration=2.002188762 podStartE2EDuration="2.002188762s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:12:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:12:05.001091406 +0000 UTC m=+1449.775589745" watchObservedRunningTime="2026-09-14 08:12:05.002188762 +0000 UTC m=+1449.776687096"
Sep 14 08:12:14.097752 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:14.097721    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:12:15.990601 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:15.990569    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:12:27.827885 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:27.827804    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:12:27.828574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:27.828042    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" podUID="98e8625a-d408-41b6-9d45-f2bd151f6b32" containerName="maas-api" containerID="cri-o://7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7" gracePeriod=30
Sep 14 08:12:28.093032 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.092973    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:12:28.134628 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.134591    2577 generic.go:356] "Generic (PLEG): container finished" podID="98e8625a-d408-41b6-9d45-f2bd151f6b32" containerID="7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7" exitCode=0
Sep 14 08:12:28.134806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.134654    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" event={"ID":"98e8625a-d408-41b6-9d45-f2bd151f6b32","Type":"ContainerDied","Data":"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"}
Sep 14 08:12:28.134806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.134665    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"
Sep 14 08:12:28.134806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.134733    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647" event={"ID":"98e8625a-d408-41b6-9d45-f2bd151f6b32","Type":"ContainerDied","Data":"ce5e39445b617f62f702e254c7bf05ef39380c8af9ea65d6623a6d4028183e4a"}
Sep 14 08:12:28.134806 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.134759    2577 scope.go:117] "RemoveContainer" containerID="7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"
Sep 14 08:12:28.145081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.145052    2577 scope.go:117] "RemoveContainer" containerID="7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"
Sep 14 08:12:28.145432 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:12:28.145410    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7\": container with ID starting with 7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7 not found: ID does not exist" containerID="7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"
Sep 14 08:12:28.145515 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.145441    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7"} err="failed to get container status \"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7\": rpc error: code = NotFound desc = could not find container \"7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7\": container with ID starting with 7476f2aabeb4c7dbda69b7054a3c825600a6299d9664132d889d35bee8dc75f7 not found: ID does not exist"
Sep 14 08:12:28.228640 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.228607    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9csc\" (UniqueName: \"kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc\") pod \"98e8625a-d408-41b6-9d45-f2bd151f6b32\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") "
Sep 14 08:12:28.228955 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.228910    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls\") pod \"98e8625a-d408-41b6-9d45-f2bd151f6b32\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") "
Sep 14 08:12:28.229069 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.229057    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs\") pod \"98e8625a-d408-41b6-9d45-f2bd151f6b32\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") "
Sep 14 08:12:28.229256 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.229244    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca\") pod \"98e8625a-d408-41b6-9d45-f2bd151f6b32\" (UID: \"98e8625a-d408-41b6-9d45-f2bd151f6b32\") "
Sep 14 08:12:28.230004 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.229976    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98e8625a-d408-41b6-9d45-f2bd151f6b32" (UID: "98e8625a-d408-41b6-9d45-f2bd151f6b32"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:28.232359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.232310    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98e8625a-d408-41b6-9d45-f2bd151f6b32" (UID: "98e8625a-d408-41b6-9d45-f2bd151f6b32"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:28.232359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.232312    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc" (OuterVolumeSpecName: "kube-api-access-x9csc") pod "98e8625a-d408-41b6-9d45-f2bd151f6b32" (UID: "98e8625a-d408-41b6-9d45-f2bd151f6b32"). InnerVolumeSpecName "kube-api-access-x9csc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:28.234507 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.234473    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "98e8625a-d408-41b6-9d45-f2bd151f6b32" (UID: "98e8625a-d408-41b6-9d45-f2bd151f6b32"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:28.330176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.330141    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.330176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.330170    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/98e8625a-d408-41b6-9d45-f2bd151f6b32-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.330176 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.330180    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98e8625a-d408-41b6-9d45-f2bd151f6b32-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.330412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.330191    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9csc\" (UniqueName: \"kubernetes.io/projected/98e8625a-d408-41b6-9d45-f2bd151f6b32-kube-api-access-x9csc\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:28.461433 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.461399    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:12:28.466049 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:28.466024    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w4-b69a34-75856d86cd-rm647"]
Sep 14 08:12:29.116514 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.116479    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" podUID="6ca66dec-ac84-4a06-bd60-616ba55963be" containerName="main" containerID="cri-o://976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0" gracePeriod=45
Sep 14 08:12:29.369347 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.369280    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:12:29.442810 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.442776    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgq76\" (UniqueName: \"kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.442907    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443015 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443142 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443046    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443142 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443075    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443243 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443140    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache\") pod \"6ca66dec-ac84-4a06-bd60-616ba55963be\" (UID: \"6ca66dec-ac84-4a06-bd60-616ba55963be\") "
Sep 14 08:12:29.443243 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443192    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.443419 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443397    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home" (OuterVolumeSpecName: "home") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.443607 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443466    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.443726 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.443632    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.445755 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.445731    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:29.445995 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.445970    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76" (OuterVolumeSpecName: "kube-api-access-vgq76") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "kube-api-access-vgq76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:29.446203 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.446183    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm" (OuterVolumeSpecName: "dshm") pod "6ca66dec-ac84-4a06-bd60-616ba55963be" (UID: "6ca66dec-ac84-4a06-bd60-616ba55963be"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:29.544703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.544667    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ca66dec-ac84-4a06-bd60-616ba55963be-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.544703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.544694    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.544703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.544706    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.544942 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.544718    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgq76\" (UniqueName: \"kubernetes.io/projected/6ca66dec-ac84-4a06-bd60-616ba55963be-kube-api-access-vgq76\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.544942 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.544729    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ca66dec-ac84-4a06-bd60-616ba55963be-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:29.762305 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:29.762273    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98e8625a-d408-41b6-9d45-f2bd151f6b32" path="/var/lib/kubelet/pods/98e8625a-d408-41b6-9d45-f2bd151f6b32/volumes"
Sep 14 08:12:30.148227 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.148196    2577 generic.go:356] "Generic (PLEG): container finished" podID="6ca66dec-ac84-4a06-bd60-616ba55963be" containerID="976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0" exitCode=0
Sep 14 08:12:30.148227 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.148230    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"
Sep 14 08:12:30.150291 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.148275    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" event={"ID":"6ca66dec-ac84-4a06-bd60-616ba55963be","Type":"ContainerDied","Data":"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"}
Sep 14 08:12:30.150291 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.148314    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g" event={"ID":"6ca66dec-ac84-4a06-bd60-616ba55963be","Type":"ContainerDied","Data":"fd9c6a4f0d13f2ea406308f7ccd0765754b67776d22b38cbd988129f9bd43ba7"}
Sep 14 08:12:30.150291 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.148332    2577 scope.go:117] "RemoveContainer" containerID="976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"
Sep 14 08:12:30.157449 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.157432    2577 scope.go:117] "RemoveContainer" containerID="976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"
Sep 14 08:12:30.157732 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:12:30.157713    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0\": container with ID starting with 976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0 not found: ID does not exist" containerID="976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"
Sep 14 08:12:30.157821 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.157743    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0"} err="failed to get container status \"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0\": rpc error: code = NotFound desc = could not find container \"976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0\": container with ID starting with 976665683de94f4e44931115419c4b564f15b3cc0c40548d050e8059cf1475e0 not found: ID does not exist"
Sep 14 08:12:30.165760 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.165732    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:12:30.167787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:30.167767    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-b69a34/premium-w4-b69a34-kserve-54bb54759f-l5q6g"]
Sep 14 08:12:31.156824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.156792    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerID="a11da3e0aa04d830f743b3f10c246fa45456bc8965280ea398c16866e04a7208" exitCode=0
Sep 14 08:12:31.157285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.156841    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerDied","Data":"a11da3e0aa04d830f743b3f10c246fa45456bc8965280ea398c16866e04a7208"}
Sep 14 08:12:31.157285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.157221    2577 scope.go:117] "RemoveContainer" containerID="a11da3e0aa04d830f743b3f10c246fa45456bc8965280ea398c16866e04a7208"
Sep 14 08:12:31.158377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.158349    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:12:31.158445 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.158417    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:12:31.158485 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.158447    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:12:31.760440 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:31.760411    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ca66dec-ac84-4a06-bd60-616ba55963be" path="/var/lib/kubelet/pods/6ca66dec-ac84-4a06-bd60-616ba55963be/volumes"
Sep 14 08:12:32.172282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:32.172236    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerStarted","Data":"43d319e5fc0fc8fe222384176963e33ee5181df503ee3b345597fa5f3764c0cf"}
Sep 14 08:12:32.320188 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:32.320152    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:12:32.320412 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:32.320390    2577 patch_prober.go:28] interesting pod/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:12:32.320473 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:32.320443    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 14 08:12:33.320534 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:33.320495    2577 patch_prober.go:28] interesting pod/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:12:33.320956 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:33.320563    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 14 08:12:34.320083 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:34.320044    2577 patch_prober.go:28] interesting pod/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:12:34.320317 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:34.320112    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 14 08:12:34.896940 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:34.894576    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:12:34.896940 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:34.894833    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy" containerID="cri-o://0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8" gracePeriod=30
Sep 14 08:12:35.320836 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:35.320801    2577 patch_prober.go:28] interesting pod/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:12:35.321035 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:35.320869    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 14 08:12:36.320658 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:36.320620    2577 patch_prober.go:28] interesting pod/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 14 08:12:36.321044 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:36.320674    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 14 08:12:37.323983 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:37.323952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:12:37.324391 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:37.324293    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:12:37.324890 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:37.324875    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:12:38.871706 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:38.871674    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:12:38.872152 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:38.871934    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" podUID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" containerName="maas-api" containerID="cri-o://590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d" gracePeriod=30
Sep 14 08:12:39.133643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.133589    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:12:39.153422 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.153400    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-skp25\" (UniqueName: \"kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25\") pod \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") "
Sep 14 08:12:39.153550 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.153486    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") pod \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") "
Sep 14 08:12:39.153618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.153553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs\") pod \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") "
Sep 14 08:12:39.153618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.153585    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca\") pod \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\" (UID: \"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21\") "
Sep 14 08:12:39.154042 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.154013    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" (UID: "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:39.155616 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.155593    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" (UID: "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:39.155720 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.155636    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25" (OuterVolumeSpecName: "kube-api-access-skp25") pod "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" (UID: "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21"). InnerVolumeSpecName "kube-api-access-skp25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:39.155842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.155828    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" (UID: "b0d9a99a-902f-4fa4-a1b6-a58e3f848d21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:12:39.221384 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.221359    2577 generic.go:356] "Generic (PLEG): container finished" podID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" containerID="590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d" exitCode=0
Sep 14 08:12:39.221500 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.221403    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"
Sep 14 08:12:39.221500 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.221447    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" event={"ID":"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21","Type":"ContainerDied","Data":"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"}
Sep 14 08:12:39.221574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.221519    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc" event={"ID":"b0d9a99a-902f-4fa4-a1b6-a58e3f848d21","Type":"ContainerDied","Data":"15b9b00cef952b44e1f03171125d2afa26ea171ab93c4abdc718e0e00b6b330f"}
Sep 14 08:12:39.221574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.221546    2577 scope.go:117] "RemoveContainer" containerID="590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"
Sep 14 08:12:39.232112 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.232098    2577 scope.go:117] "RemoveContainer" containerID="590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"
Sep 14 08:12:39.232444 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:12:39.232426    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d\": container with ID starting with 590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d not found: ID does not exist" containerID="590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"
Sep 14 08:12:39.232503 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.232453    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d"} err="failed to get container status \"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d\": rpc error: code = NotFound desc = could not find container \"590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d\": container with ID starting with 590ea3f7435217ea383ce9836a8dd484f7cbdb89a8ece21973e15ef956c86f3d not found: ID does not exist"
Sep 14 08:12:39.245011 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.244991    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:12:39.248746 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.248724    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-b914f7-6b99474756-9pcgc"]
Sep 14 08:12:39.255233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.255138    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-skp25\" (UniqueName: \"kubernetes.io/projected/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-kube-api-access-skp25\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.255233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.255165    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.255233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.255181    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.255233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.255195    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:39.761452 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:39.761419    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" path="/var/lib/kubelet/pods/b0d9a99a-902f-4fa4-a1b6-a58e3f848d21/volumes"
Sep 14 08:12:40.158638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.158615    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:12:40.228608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.228575    2577 generic.go:356] "Generic (PLEG): container finished" podID="76293e7e-c869-4277-812e-4c3439552980" containerID="0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8" exitCode=0
Sep 14 08:12:40.228791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.228626    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"
Sep 14 08:12:40.228791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.228656    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerDied","Data":"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"}
Sep 14 08:12:40.228791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.228708    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm" event={"ID":"76293e7e-c869-4277-812e-4c3439552980","Type":"ContainerDied","Data":"81d4627b01f19143d469086a5d201b619deb9f8bac748e23531041f86d296264"}
Sep 14 08:12:40.228791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.228736    2577 scope.go:117] "RemoveContainer" containerID="0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"
Sep 14 08:12:40.237567 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.237549    2577 scope.go:117] "RemoveContainer" containerID="ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"
Sep 14 08:12:40.245491 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.245472    2577 scope.go:117] "RemoveContainer" containerID="0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"
Sep 14 08:12:40.245773 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:12:40.245755    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8\": container with ID starting with 0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8 not found: ID does not exist" containerID="0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"
Sep 14 08:12:40.245842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.245778    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8"} err="failed to get container status \"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8\": rpc error: code = NotFound desc = could not find container \"0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8\": container with ID starting with 0b54d82f7e25adca927305d0bd8e2fa3af7fb0c6561ac12fdfc9a9263c50f0d8 not found: ID does not exist"
Sep 14 08:12:40.245842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.245793    2577 scope.go:117] "RemoveContainer" containerID="ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"
Sep 14 08:12:40.246054 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:12:40.246021    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4\": container with ID starting with ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4 not found: ID does not exist" containerID="ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"
Sep 14 08:12:40.246119 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.246060    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4"} err="failed to get container status \"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4\": rpc error: code = NotFound desc = could not find container \"ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4\": container with ID starting with ca890f3756ef20d514a016e8a66cd713d02a459c17a9a6b4834a9079e2e8bba4 not found: ID does not exist"
Sep 14 08:12:40.264441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264423    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264470    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fjpnh\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264489    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264543    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264579    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264747 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264652    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264747 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264707    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264854 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264761    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264854 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264791    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data\") pod \"76293e7e-c869-4277-812e-4c3439552980\" (UID: \"76293e7e-c869-4277-812e-4c3439552980\") "
Sep 14 08:12:40.264854 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264815    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:40.265065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264853    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:40.265065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264901    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:40.265065 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.264944    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:12:40.265222 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265120    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data" (OuterVolumeSpecName: "istio-data") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:40.265282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265229    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.265282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265247    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.265282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265257    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.265282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265266    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/76293e7e-c869-4277-812e-4c3439552980-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.265282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.265278    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.266611 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.266585    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh" (OuterVolumeSpecName: "kube-api-access-fjpnh") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "kube-api-access-fjpnh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:40.266745 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.266730    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token" (OuterVolumeSpecName: "istio-token") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:12:40.267195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.267171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:12:40.267277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.267170    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "76293e7e-c869-4277-812e-4c3439552980" (UID: "76293e7e-c869-4277-812e-4c3439552980"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:12:40.366505 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.366474    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.366505 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.366498    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/76293e7e-c869-4277-812e-4c3439552980-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.366505 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.366506    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/76293e7e-c869-4277-812e-4c3439552980-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.366710 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.366515    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fjpnh\" (UniqueName: \"kubernetes.io/projected/76293e7e-c869-4277-812e-4c3439552980-kube-api-access-fjpnh\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:12:40.552767 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.552741    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:12:40.563501 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:40.563478    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-b69a34-openshift-default-6b5556d64b-m7vhm"]
Sep 14 08:12:41.761290 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:41.761256    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76293e7e-c869-4277-812e-4c3439552980" path="/var/lib/kubelet/pods/76293e7e-c869-4277-812e-4c3439552980/volumes"
Sep 14 08:12:55.792863 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:55.792835    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:12:55.797669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:12:55.797648    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:13:01.689879 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:01.689835    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:13:01.690472 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:01.690144    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" podUID="d2960c4f-ac58-4145-9ef4-293864290833" containerName="payload-pre-processing" containerID="cri-o://e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8" gracePeriod=30
Sep 14 08:13:01.941551 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:01.941490    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:13:02.068864 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.068834    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume\") pod \"d2960c4f-ac58-4145-9ef4-293864290833\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") "
Sep 14 08:13:02.069056 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.068881    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxpp7\" (UniqueName: \"kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7\") pod \"d2960c4f-ac58-4145-9ef4-293864290833\" (UID: \"d2960c4f-ac58-4145-9ef4-293864290833\") "
Sep 14 08:13:02.069234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.069212    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d2960c4f-ac58-4145-9ef4-293864290833" (UID: "d2960c4f-ac58-4145-9ef4-293864290833"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:02.071095 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.071058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7" (OuterVolumeSpecName: "kube-api-access-sxpp7") pod "d2960c4f-ac58-4145-9ef4-293864290833" (UID: "d2960c4f-ac58-4145-9ef4-293864290833"). InnerVolumeSpecName "kube-api-access-sxpp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:02.169944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.169888    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d2960c4f-ac58-4145-9ef4-293864290833-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.169944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.169932    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxpp7\" (UniqueName: \"kubernetes.io/projected/d2960c4f-ac58-4145-9ef4-293864290833-kube-api-access-sxpp7\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:02.345968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.345852    2577 generic.go:356] "Generic (PLEG): container finished" podID="d2960c4f-ac58-4145-9ef4-293864290833" containerID="e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8" exitCode=0
Sep 14 08:13:02.345968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.345893    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" event={"ID":"d2960c4f-ac58-4145-9ef4-293864290833","Type":"ContainerDied","Data":"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"}
Sep 14 08:13:02.345968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.345908    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"
Sep 14 08:13:02.345968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.345963    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6" event={"ID":"d2960c4f-ac58-4145-9ef4-293864290833","Type":"ContainerDied","Data":"eda8b5110e973a28a8fc808b0d83c3d43fc6b777beaf383a55fb722c1a2a507d"}
Sep 14 08:13:02.346290 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.345984    2577 scope.go:117] "RemoveContainer" containerID="e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"
Sep 14 08:13:02.354953 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.354902    2577 scope.go:117] "RemoveContainer" containerID="e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"
Sep 14 08:13:02.355213 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:02.355195    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8\": container with ID starting with e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8 not found: ID does not exist" containerID="e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"
Sep 14 08:13:02.355264 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.355221    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8"} err="failed to get container status \"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8\": rpc error: code = NotFound desc = could not find container \"e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8\": container with ID starting with e8b2b3f25cfd9428bd010497ac3449a39853d17646620d8fbd34fb23eceda6d8 not found: ID does not exist"
Sep 14 08:13:02.368649 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.368623    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:13:02.370432 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:02.370415    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-87b7cb-54bf6d6b94-wm8d6"]
Sep 14 08:13:03.759995 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:03.759959    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2960c4f-ac58-4145-9ef4-293864290833" path="/var/lib/kubelet/pods/d2960c4f-ac58-4145-9ef4-293864290833/volumes"
Sep 14 08:13:08.187520 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:08.187488    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:13:08.188021 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:08.187775    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy" containerID="cri-o://43d319e5fc0fc8fe222384176963e33ee5181df503ee3b345597fa5f3764c0cf" gracePeriod=30
Sep 14 08:13:13.408458 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.408425    2577 generic.go:356] "Generic (PLEG): container finished" podID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerID="43d319e5fc0fc8fe222384176963e33ee5181df503ee3b345597fa5f3764c0cf" exitCode=0
Sep 14 08:13:13.408881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.408485    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerDied","Data":"43d319e5fc0fc8fe222384176963e33ee5181df503ee3b345597fa5f3764c0cf"}
Sep 14 08:13:13.408881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.408551    2577 scope.go:117] "RemoveContainer" containerID="a11da3e0aa04d830f743b3f10c246fa45456bc8965280ea398c16866e04a7208"
Sep 14 08:13:13.441165 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.441137    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:13:13.471588 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471524    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471588 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471587    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471770 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471609    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471770 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471630    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471770 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471648    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmz58\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471770 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471673    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471981 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471792    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:13.471981 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.471981 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.471891    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:13.472138 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472121    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:13.472230 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472211    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.472285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472235    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.472285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472249    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ff6544eb-c57a-41e1-921c-7327168d38fd-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.472397 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472296    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:13.472397 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.472313    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data" (OuterVolumeSpecName: "istio-data") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:13.473953 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.473904    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58" (OuterVolumeSpecName: "kube-api-access-wmz58") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "kube-api-access-wmz58". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:13.474096 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.474078    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:13:13.572615 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.572584    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.572776 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.572622    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token\") pod \"ff6544eb-c57a-41e1-921c-7327168d38fd\" (UID: \"ff6544eb-c57a-41e1-921c-7327168d38fd\") "
Sep 14 08:13:13.572975 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.572958    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.573043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.572979    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmz58\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-kube-api-access-wmz58\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.573043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.572993    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.573043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.573005    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.574496 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.574475    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:13.574598 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.574579    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token" (OuterVolumeSpecName: "istio-token") pod "ff6544eb-c57a-41e1-921c-7327168d38fd" (UID: "ff6544eb-c57a-41e1-921c-7327168d38fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:13.673458 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.673423    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:13.673458 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:13.673451    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ff6544eb-c57a-41e1-921c-7327168d38fd-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:14.415066 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:14.415035    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"
Sep 14 08:13:14.415510 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:14.415084    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7" event={"ID":"ff6544eb-c57a-41e1-921c-7327168d38fd","Type":"ContainerDied","Data":"9617c2e104403fe085395b157b59b6c7e04c78d21cc5fc942149e77e8e54c042"}
Sep 14 08:13:14.415510 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:14.415146    2577 scope.go:117] "RemoveContainer" containerID="43d319e5fc0fc8fe222384176963e33ee5181df503ee3b345597fa5f3764c0cf"
Sep 14 08:13:14.432813 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:14.432785    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:13:14.438441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:14.438418    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-87b7cb-openshift-default-5996dfd876-wgbv7"]
Sep 14 08:13:15.760391 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:15.760362    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" path="/var/lib/kubelet/pods/ff6544eb-c57a-41e1-921c-7327168d38fd/volumes"
Sep 14 08:13:16.653109 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653078    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7864-975b27"]
Sep 14 08:13:16.653462 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653450    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ca66dec-ac84-4a06-bd60-616ba55963be" containerName="main"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653471    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ca66dec-ac84-4a06-bd60-616ba55963be" containerName="main"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653483    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98e8625a-d408-41b6-9d45-f2bd151f6b32" containerName="maas-api"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653490    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="98e8625a-d408-41b6-9d45-f2bd151f6b32" containerName="maas-api"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653500    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653506    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653518 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653518    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653524    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653531    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" containerName="maas-api"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653535    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" containerName="maas-api"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653542    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653547    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653554    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653560    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653566    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2960c4f-ac58-4145-9ef4-293864290833" containerName="payload-pre-processing"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653571    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2960c4f-ac58-4145-9ef4-293864290833" containerName="payload-pre-processing"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653633    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653642    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653648    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="76293e7e-c869-4277-812e-4c3439552980" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653654    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="98e8625a-d408-41b6-9d45-f2bd151f6b32" containerName="maas-api"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653660    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff6544eb-c57a-41e1-921c-7327168d38fd" containerName="istio-proxy"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653666    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ca66dec-ac84-4a06-bd60-616ba55963be" containerName="main"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653672    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0d9a99a-902f-4fa4-a1b6-a58e3f848d21" containerName="maas-api"
Sep 14 08:13:16.653705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.653677    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2960c4f-ac58-4145-9ef4-293864290833" containerName="payload-pre-processing"
Sep 14 08:13:16.656133 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.656118    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:16.658573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.658552    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fvblv\""
Sep 14 08:13:16.664086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.664067    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-975b27"]
Sep 14 08:13:16.694077 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.694042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9jxt\" (UniqueName: \"kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt\") pod \"test-curl-7864-975b27\" (UID: \"87d82086-0295-43a9-b31c-f3aa5714e768\") " pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:16.795304 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.795268    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9jxt\" (UniqueName: \"kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt\") pod \"test-curl-7864-975b27\" (UID: \"87d82086-0295-43a9-b31c-f3aa5714e768\") " pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:16.803191 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.803159    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9jxt\" (UniqueName: \"kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt\") pod \"test-curl-7864-975b27\" (UID: \"87d82086-0295-43a9-b31c-f3aa5714e768\") " pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:16.966096 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:16.966012    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:17.237224 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:17.237202    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-975b27"]
Sep 14 08:13:17.431856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:17.431825    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-975b27" event={"ID":"87d82086-0295-43a9-b31c-f3aa5714e768","Type":"ContainerStarted","Data":"da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f"}
Sep 14 08:13:17.431856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:17.431867    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-975b27" event={"ID":"87d82086-0295-43a9-b31c-f3aa5714e768","Type":"ContainerStarted","Data":"abbc308554b73a945c1a06e06d3f6265b5bd98242765766af9d777a00e7e812d"}
Sep 14 08:13:17.447889 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:17.447846    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7864-975b27" podStartSLOduration=1.447831607 podStartE2EDuration="1.447831607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:17.447223562 +0000 UTC m=+1522.221721898" watchObservedRunningTime="2026-09-14 08:13:17.447831607 +0000 UTC m=+1522.222329942"
Sep 14 08:13:18.052442 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:18.052408    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7864-975b27"]
Sep 14 08:13:18.054462 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:18.054437    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7864-975b27"]
Sep 14 08:13:18.437345 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:18.437319    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7864-975b27" container="test-curl-7864-975b27"
Sep 14 08:13:19.445207 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.445146    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7864-975b27" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" containerName="test-curl-7864-975b27" containerID="cri-o://da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f" gracePeriod=2
Sep 14 08:13:19.447846 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:19.447808    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7864-975b27\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:19.454282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.454258    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:13:19.456847 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:19.456822    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7864-975b27\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:19.568675 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.568641    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:13:19.568932 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.568885    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" podUID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" containerName="maas-api" containerID="cri-o://9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9" gracePeriod=30
Sep 14 08:13:19.593576 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:19.593544    2577 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7864-975b27\" is forbidden: User \"system:node:ip-10-0-128-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-128-30.ec2.internal' and this object" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:19.811734 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.811712    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:13:19.821672 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.821649    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca\") pod \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") "
Sep 14 08:13:19.821784 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.821683    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs\") pod \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") "
Sep 14 08:13:19.821784 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.821751    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tlm57\" (UniqueName: \"kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57\") pod \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") "
Sep 14 08:13:19.821857 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.821823    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") pod \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\" (UID: \"ae8554b9-efdf-4b7c-9126-5d72a52ee54f\") "
Sep 14 08:13:19.822046 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.822024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ae8554b9-efdf-4b7c-9126-5d72a52ee54f" (UID: "ae8554b9-efdf-4b7c-9126-5d72a52ee54f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:19.822160 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.822146    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:19.823791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.823758    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ae8554b9-efdf-4b7c-9126-5d72a52ee54f" (UID: "ae8554b9-efdf-4b7c-9126-5d72a52ee54f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:19.823791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.823780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57" (OuterVolumeSpecName: "kube-api-access-tlm57") pod "ae8554b9-efdf-4b7c-9126-5d72a52ee54f" (UID: "ae8554b9-efdf-4b7c-9126-5d72a52ee54f"). InnerVolumeSpecName "kube-api-access-tlm57". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:19.824019 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.823808    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ae8554b9-efdf-4b7c-9126-5d72a52ee54f" (UID: "ae8554b9-efdf-4b7c-9126-5d72a52ee54f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:19.923246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.923213    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:19.923246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.923239    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tlm57\" (UniqueName: \"kubernetes.io/projected/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-kube-api-access-tlm57\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:19.923246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:19.923249    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ae8554b9-efdf-4b7c-9126-5d72a52ee54f-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:20.451527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.451490    2577 generic.go:356] "Generic (PLEG): container finished" podID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" containerID="9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9" exitCode=0
Sep 14 08:13:20.452034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.451540    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"
Sep 14 08:13:20.452034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.451575    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" event={"ID":"ae8554b9-efdf-4b7c-9126-5d72a52ee54f","Type":"ContainerDied","Data":"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"}
Sep 14 08:13:20.452034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.451627    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm" event={"ID":"ae8554b9-efdf-4b7c-9126-5d72a52ee54f","Type":"ContainerDied","Data":"2e44bb39a0646b70f244b8c45cd905484eff1df040403e8fa4f320b198fd03a6"}
Sep 14 08:13:20.452034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.451649    2577 scope.go:117] "RemoveContainer" containerID="9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"
Sep 14 08:13:20.460908 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.460890    2577 scope.go:117] "RemoveContainer" containerID="9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"
Sep 14 08:13:20.461207 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:20.461187    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9\": container with ID starting with 9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9 not found: ID does not exist" containerID="9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"
Sep 14 08:13:20.461254 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.461216    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9"} err="failed to get container status \"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9\": rpc error: code = NotFound desc = could not find container \"9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9\": container with ID starting with 9b8a909e09f5dc2a96aeb8ee96c163ea6c9226a18bfcb5a7b8d0d134c7100dc9 not found: ID does not exist"
Sep 14 08:13:20.473128 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.473100    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:13:20.476309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:20.476291    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-b990f4-65d47c7cd6-wfglm"]
Sep 14 08:13:21.625662 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.625635    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:21.639288 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.639261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r9jxt\" (UniqueName: \"kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt\") pod \"87d82086-0295-43a9-b31c-f3aa5714e768\" (UID: \"87d82086-0295-43a9-b31c-f3aa5714e768\") "
Sep 14 08:13:21.641141 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.641114    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt" (OuterVolumeSpecName: "kube-api-access-r9jxt") pod "87d82086-0295-43a9-b31c-f3aa5714e768" (UID: "87d82086-0295-43a9-b31c-f3aa5714e768"). InnerVolumeSpecName "kube-api-access-r9jxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:21.741013 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.740935    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r9jxt\" (UniqueName: \"kubernetes.io/projected/87d82086-0295-43a9-b31c-f3aa5714e768-kube-api-access-r9jxt\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:21.760808 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.760780    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" path="/var/lib/kubelet/pods/87d82086-0295-43a9-b31c-f3aa5714e768/volumes"
Sep 14 08:13:21.761240 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:21.761223    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" path="/var/lib/kubelet/pods/ae8554b9-efdf-4b7c-9126-5d72a52ee54f/volumes"
Sep 14 08:13:22.464616 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:22.464581    2577 generic.go:356] "Generic (PLEG): container finished" podID="87d82086-0295-43a9-b31c-f3aa5714e768" containerID="da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f" exitCode=137
Sep 14 08:13:22.464816 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:22.464628    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-975b27"
Sep 14 08:13:22.464816 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:22.464656    2577 scope.go:117] "RemoveContainer" containerID="da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f"
Sep 14 08:13:22.473357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:22.473339    2577 scope.go:117] "RemoveContainer" containerID="da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f"
Sep 14 08:13:22.473620 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:22.473598    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f\": container with ID starting with da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f not found: ID does not exist" containerID="da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f"
Sep 14 08:13:22.473686 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:22.473628    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f"} err="failed to get container status \"da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f\": rpc error: code = NotFound desc = could not find container \"da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f\": container with ID starting with da7c12649a21a604e0eac00302a6e2340ee2f0dd1ef3f64a9d9a8513cd3f477f not found: ID does not exist"
Sep 14 08:13:28.296666 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.296635    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:13:28.297115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297099    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" containerName="test-curl-7864-975b27"
Sep 14 08:13:28.297162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297119    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" containerName="test-curl-7864-975b27"
Sep 14 08:13:28.297162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297147    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" containerName="maas-api"
Sep 14 08:13:28.297162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297153    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" containerName="maas-api"
Sep 14 08:13:28.297269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297236    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae8554b9-efdf-4b7c-9126-5d72a52ee54f" containerName="maas-api"
Sep 14 08:13:28.297269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.297254    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="87d82086-0295-43a9-b31c-f3aa5714e768" containerName="test-curl-7864-975b27"
Sep 14 08:13:28.300260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.300245    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.303294 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.303275    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"sub-test-model-7746e4-kserve-self-signed-certs\""
Sep 14 08:13:28.311267 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.311245    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:13:28.403834 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.403803    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qws2t\" (UniqueName: \"kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.403834 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.403834    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.404061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.403867    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.404061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.403887    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.404061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.403973    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.404171 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.404075    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505168    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505362 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505215    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qws2t\" (UniqueName: \"kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505362 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505279    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505362 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505529 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505367    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505576 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505524    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505716    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505802 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505744    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.505802 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.505784    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.507687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.507654    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.507794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.507777    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.513269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.513250    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qws2t\" (UniqueName: \"kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t\") pod \"sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.610757 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.610684    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:28.886068 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:28.885803    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:13:29.501559 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:29.501520    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" event={"ID":"9e19cb79-fb50-4e0c-8f74-178264461f32","Type":"ContainerStarted","Data":"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"}
Sep 14 08:13:29.501969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:29.501568    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" event={"ID":"9e19cb79-fb50-4e0c-8f74-178264461f32","Type":"ContainerStarted","Data":"9261d82933285578030c96dacfa1cc3d15b661106ab9d56bac0f9add3d88be0b"}
Sep 14 08:13:29.520999 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:29.520953    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" podStartSLOduration=1.520937046 podStartE2EDuration="1.520937046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:13:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:13:29.519753314 +0000 UTC m=+1534.294251650" watchObservedRunningTime="2026-09-14 08:13:29.520937046 +0000 UTC m=+1534.295435372"
Sep 14 08:13:34.472909 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.472824    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" podUID="559d7414-df05-4992-8fc3-1749abc7152b" containerName="main" containerID="cri-o://ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752" gracePeriod=45
Sep 14 08:13:34.720000 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.719975    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:13:34.863729 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863639    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.863903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.863903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863809    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.863903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863837    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qsdx8\" (UniqueName: \"kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.863903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863862    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.863903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.863894    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir\") pod \"559d7414-df05-4992-8fc3-1749abc7152b\" (UID: \"559d7414-df05-4992-8fc3-1749abc7152b\") "
Sep 14 08:13:34.864196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.864109    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home" (OuterVolumeSpecName: "home") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:34.864196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.864167    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache" (OuterVolumeSpecName: "model-cache") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:34.864282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.864227    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:34.864337 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.864290    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:34.864337 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.864310    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:34.866055 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.866024    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm" (OuterVolumeSpecName: "dshm") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:13:34.866055 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.866047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:34.866205 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.866125    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8" (OuterVolumeSpecName: "kube-api-access-qsdx8") pod "559d7414-df05-4992-8fc3-1749abc7152b" (UID: "559d7414-df05-4992-8fc3-1749abc7152b"). InnerVolumeSpecName "kube-api-access-qsdx8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:34.965750 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.965717    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:34.965750 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.965747    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qsdx8\" (UniqueName: \"kubernetes.io/projected/559d7414-df05-4992-8fc3-1749abc7152b-kube-api-access-qsdx8\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:34.965750 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.965759    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/559d7414-df05-4992-8fc3-1749abc7152b-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:34.966014 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:34.965768    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/559d7414-df05-4992-8fc3-1749abc7152b-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:35.535641 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.535610    2577 generic.go:356] "Generic (PLEG): container finished" podID="559d7414-df05-4992-8fc3-1749abc7152b" containerID="ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752" exitCode=0
Sep 14 08:13:35.536116 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.535654    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"
Sep 14 08:13:35.536116 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.535657    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" event={"ID":"559d7414-df05-4992-8fc3-1749abc7152b","Type":"ContainerDied","Data":"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"}
Sep 14 08:13:35.536116 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.535697    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j" event={"ID":"559d7414-df05-4992-8fc3-1749abc7152b","Type":"ContainerDied","Data":"48d9e086841dfd869a0983cbe80c852c03ec3ea61e3e26e38d4c0a2fd694feed"}
Sep 14 08:13:35.536116 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.535718    2577 scope.go:117] "RemoveContainer" containerID="ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"
Sep 14 08:13:35.545390 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.545373    2577 scope.go:117] "RemoveContainer" containerID="ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"
Sep 14 08:13:35.545654 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:35.545635    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752\": container with ID starting with ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752 not found: ID does not exist" containerID="ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"
Sep 14 08:13:35.545725 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.545663    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752"} err="failed to get container status \"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752\": rpc error: code = NotFound desc = could not find container \"ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752\": container with ID starting with ff7fdebcb939bff1f747ce10a1fdcb3a0aa6c7321178fccecbb2688abde79752 not found: ID does not exist"
Sep 14 08:13:35.559784 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.559760    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:13:35.567769 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.567750    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/auth-test-model-7746e4-kserve-7bccc6c895-9975j"]
Sep 14 08:13:35.761159 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:35.761134    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="559d7414-df05-4992-8fc3-1749abc7152b" path="/var/lib/kubelet/pods/559d7414-df05-4992-8fc3-1749abc7152b/volumes"
Sep 14 08:13:37.214326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.214297    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:13:37.214775 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.214536    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" podUID="1af13d82-c326-4696-8438-af327d1a74b9" containerName="maas-api" containerID="cri-o://88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5" gracePeriod=30
Sep 14 08:13:37.460296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.460266    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:13:37.547648 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.547558    2577 generic.go:356] "Generic (PLEG): container finished" podID="1af13d82-c326-4696-8438-af327d1a74b9" containerID="88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5" exitCode=0
Sep 14 08:13:37.547648 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.547601    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"
Sep 14 08:13:37.547848 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.547643    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" event={"ID":"1af13d82-c326-4696-8438-af327d1a74b9","Type":"ContainerDied","Data":"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"}
Sep 14 08:13:37.547848 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.547701    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns" event={"ID":"1af13d82-c326-4696-8438-af327d1a74b9","Type":"ContainerDied","Data":"f66400ff46a88cbdfe6cee57cfa02316958568fdc7f75a71edcab35e4caa8f5a"}
Sep 14 08:13:37.547848 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.547731    2577 scope.go:117] "RemoveContainer" containerID="88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"
Sep 14 08:13:37.556373 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.556354    2577 scope.go:117] "RemoveContainer" containerID="88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"
Sep 14 08:13:37.556655 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:13:37.556637    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5\": container with ID starting with 88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5 not found: ID does not exist" containerID="88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"
Sep 14 08:13:37.556703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.556668    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5"} err="failed to get container status \"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5\": rpc error: code = NotFound desc = could not find container \"88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5\": container with ID starting with 88b7ee1ec3788805dc92bd6d4a32379e5e9f2f2aabd2435d99c0b31b19899ce5 not found: ID does not exist"
Sep 14 08:13:37.589271 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.589251    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca\") pod \"1af13d82-c326-4696-8438-af327d1a74b9\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") "
Sep 14 08:13:37.589388 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.589281    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs\") pod \"1af13d82-c326-4696-8438-af327d1a74b9\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") "
Sep 14 08:13:37.589388 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.589341    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nf2xc\" (UniqueName: \"kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc\") pod \"1af13d82-c326-4696-8438-af327d1a74b9\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") "
Sep 14 08:13:37.589501 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.589443    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") pod \"1af13d82-c326-4696-8438-af327d1a74b9\" (UID: \"1af13d82-c326-4696-8438-af327d1a74b9\") "
Sep 14 08:13:37.589716 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.589687    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1af13d82-c326-4696-8438-af327d1a74b9" (UID: "1af13d82-c326-4696-8438-af327d1a74b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:13:37.591423 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.591394    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1af13d82-c326-4696-8438-af327d1a74b9" (UID: "1af13d82-c326-4696-8438-af327d1a74b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:37.591521 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.591426    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1af13d82-c326-4696-8438-af327d1a74b9" (UID: "1af13d82-c326-4696-8438-af327d1a74b9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:13:37.591573 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.591524    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc" (OuterVolumeSpecName: "kube-api-access-nf2xc") pod "1af13d82-c326-4696-8438-af327d1a74b9" (UID: "1af13d82-c326-4696-8438-af327d1a74b9"). InnerVolumeSpecName "kube-api-access-nf2xc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:13:37.690936 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.690888    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1af13d82-c326-4696-8438-af327d1a74b9-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.690936 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.690913    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.691123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.690952    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nf2xc\" (UniqueName: \"kubernetes.io/projected/1af13d82-c326-4696-8438-af327d1a74b9-kube-api-access-nf2xc\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.691123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.690964    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1af13d82-c326-4696-8438-af327d1a74b9-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:13:37.864672 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.864592    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:13:37.874569 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:37.872143    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-afcf8a-84f4654844-28tns"]
Sep 14 08:13:38.611004 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:38.610970    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:38.611454 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:38.611014    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:38.623776 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:38.623749    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:39.569027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:39.568999    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:13:39.760797 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:13:39.760763    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1af13d82-c326-4696-8438-af327d1a74b9" path="/var/lib/kubelet/pods/1af13d82-c326-4696-8438-af327d1a74b9/volumes"
Sep 14 08:14:11.001544 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.001514    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.001942    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1af13d82-c326-4696-8438-af327d1a74b9" containerName="maas-api"
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.001955    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="1af13d82-c326-4696-8438-af327d1a74b9" containerName="maas-api"
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.001967    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="559d7414-df05-4992-8fc3-1749abc7152b" containerName="main"
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.001973    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="559d7414-df05-4992-8fc3-1749abc7152b" containerName="main"
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.002040    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="1af13d82-c326-4696-8438-af327d1a74b9" containerName="maas-api"
Sep 14 08:14:11.002157 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.002049    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="559d7414-df05-4992-8fc3-1749abc7152b" containerName="main"
Sep 14 08:14:11.005142 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.005124    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.009091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.009061    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"sub-test-model-ea9d35-kserve-self-signed-certs\""
Sep 14 08:14:11.009216 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.009091    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"kube-root-ca.crt\""
Sep 14 08:14:11.009216 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.009146    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"openshift-service-ca.crt\""
Sep 14 08:14:11.009216 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.009067    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"default-dockercfg-5lrdl\""
Sep 14 08:14:11.016112 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.016091    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:14:11.088208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088174    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.088366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088238    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.088366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088264    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.088366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088285    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.088366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.088366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.088332    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5wjsx\" (UniqueName: \"kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189056 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189022    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189056 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189060    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189310 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189092    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189310 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189116    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5wjsx\" (UniqueName: \"kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189310 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189174    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189310 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189253    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189550    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189585    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.189634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.189601    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.191367 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.191348    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.191723 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.191702    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.196958 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.196940    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5wjsx\" (UniqueName: \"kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx\") pod \"sub-test-model-ea9d35-kserve-57749b6d77-mcldz\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.317132 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.317062    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:11.588956 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.588932    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:14:11.591013 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:14:11.590989    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd9d75f96_daa8_476a_8a5c_03c4b787818f.slice/crio-360e2dc72cec2e852ecbb2cbd5bb64daeddfac4f4c0322dd4c4df10cc9f3113e WatchSource:0}: Error finding container 360e2dc72cec2e852ecbb2cbd5bb64daeddfac4f4c0322dd4c4df10cc9f3113e: Status 404 returned error can't find the container with id 360e2dc72cec2e852ecbb2cbd5bb64daeddfac4f4c0322dd4c4df10cc9f3113e
Sep 14 08:14:11.716750 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.716717    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" event={"ID":"d9d75f96-daa8-476a-8a5c-03c4b787818f","Type":"ContainerStarted","Data":"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"}
Sep 14 08:14:11.716897 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.716759    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" event={"ID":"d9d75f96-daa8-476a-8a5c-03c4b787818f","Type":"ContainerStarted","Data":"360e2dc72cec2e852ecbb2cbd5bb64daeddfac4f4c0322dd4c4df10cc9f3113e"}
Sep 14 08:14:11.734747 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:11.734698    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" podStartSLOduration=1.7346838359999999 podStartE2EDuration="1.734683836s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:11.733308385 +0000 UTC m=+1576.507806729" watchObservedRunningTime="2026-09-14 08:14:11.734683836 +0000 UTC m=+1576.509182170"
Sep 14 08:14:21.317633 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:21.317590    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:21.318055 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:21.317666    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:21.329798 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:21.329777    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:21.773563 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:21.773537    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:14:47.846804 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:47.846767    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7864-decb0d"]
Sep 14 08:14:47.850520 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:47.850503    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:47.853324 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:47.853156    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-fvblv\""
Sep 14 08:14:47.854126 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:47.854103    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-decb0d"]
Sep 14 08:14:47.916515 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:47.916490    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8r6ks\" (UniqueName: \"kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks\") pod \"test-curl-7864-decb0d\" (UID: \"07f33594-be11-4abc-bcfb-5be3274e429c\") " pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:48.017315 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.017285    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8r6ks\" (UniqueName: \"kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks\") pod \"test-curl-7864-decb0d\" (UID: \"07f33594-be11-4abc-bcfb-5be3274e429c\") " pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:48.026006 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.025978    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8r6ks\" (UniqueName: \"kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks\") pod \"test-curl-7864-decb0d\" (UID: \"07f33594-be11-4abc-bcfb-5be3274e429c\") " pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:48.160959 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.160934    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:48.429966 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.429943    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-decb0d"]
Sep 14 08:14:48.432016 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:14:48.431992    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod07f33594_be11_4abc_bcfb_5be3274e429c.slice/crio-6079488ded8e7dce34b7d5c0fa14ca10902512dda993d218bfc9dc49d0187a5c WatchSource:0}: Error finding container 6079488ded8e7dce34b7d5c0fa14ca10902512dda993d218bfc9dc49d0187a5c: Status 404 returned error can't find the container with id 6079488ded8e7dce34b7d5c0fa14ca10902512dda993d218bfc9dc49d0187a5c
Sep 14 08:14:48.889883 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.889783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-decb0d" event={"ID":"07f33594-be11-4abc-bcfb-5be3274e429c","Type":"ContainerStarted","Data":"75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907"}
Sep 14 08:14:48.889883 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.889832    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-decb0d" event={"ID":"07f33594-be11-4abc-bcfb-5be3274e429c","Type":"ContainerStarted","Data":"6079488ded8e7dce34b7d5c0fa14ca10902512dda993d218bfc9dc49d0187a5c"}
Sep 14 08:14:48.904414 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:48.904371    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7864-decb0d" podStartSLOduration=1.90435515 podStartE2EDuration="1.90435515s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:48.903801393 +0000 UTC m=+1613.678299725" watchObservedRunningTime="2026-09-14 08:14:48.90435515 +0000 UTC m=+1613.678853510"
Sep 14 08:14:49.472118 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.472085    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7864-decb0d"]
Sep 14 08:14:49.478233 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.478209    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7864-decb0d"]
Sep 14 08:14:49.592559 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.592485    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7864-b6d170"]
Sep 14 08:14:49.592863 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.592851    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07f33594-be11-4abc-bcfb-5be3274e429c" containerName="test-curl-7864-decb0d"
Sep 14 08:14:49.592906 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.592865    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="07f33594-be11-4abc-bcfb-5be3274e429c" containerName="test-curl-7864-decb0d"
Sep 14 08:14:49.592982 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.592971    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="07f33594-be11-4abc-bcfb-5be3274e429c" containerName="test-curl-7864-decb0d"
Sep 14 08:14:49.596153 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.596137    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:49.601951 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.601887    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-b6d170"]
Sep 14 08:14:49.631864 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.631835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g7plt\" (UniqueName: \"kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt\") pod \"test-curl-7864-b6d170\" (UID: \"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570\") " pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:49.732828 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.732802    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g7plt\" (UniqueName: \"kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt\") pod \"test-curl-7864-b6d170\" (UID: \"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570\") " pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:49.741732 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.741715    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g7plt\" (UniqueName: \"kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt\") pod \"test-curl-7864-b6d170\" (UID: \"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570\") " pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:49.894823 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:14:49.894742    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7864-decb0d" container="test-curl-7864-decb0d"
Sep 14 08:14:49.907160 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:49.907142    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:50.174282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:50.174253    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7864-b6d170"]
Sep 14 08:14:50.177903 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:14:50.177878    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4fab49e_b6ce_40a0_a6b5_c3ac601c9570.slice/crio-bbc7d47ea1bd517b49ea36ea402dfd1526bb2a0ced29c0043241d5233abb05fd WatchSource:0}: Error finding container bbc7d47ea1bd517b49ea36ea402dfd1526bb2a0ced29c0043241d5233abb05fd: Status 404 returned error can't find the container with id bbc7d47ea1bd517b49ea36ea402dfd1526bb2a0ced29c0043241d5233abb05fd
Sep 14 08:14:50.900426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:50.900378    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-b6d170" event={"ID":"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570","Type":"ContainerStarted","Data":"aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5"}
Sep 14 08:14:50.900426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:50.900430    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7864-b6d170" event={"ID":"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570","Type":"ContainerStarted","Data":"bbc7d47ea1bd517b49ea36ea402dfd1526bb2a0ced29c0043241d5233abb05fd"}
Sep 14 08:14:50.900954 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:50.900463    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7864-decb0d" podUID="07f33594-be11-4abc-bcfb-5be3274e429c" containerName="test-curl-7864-decb0d" containerID="cri-o://75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907" gracePeriod=2
Sep 14 08:14:50.915931 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:50.915870    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7864-b6d170" podStartSLOduration=1.91585362 podStartE2EDuration="1.91585362s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:14:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:14:50.914527126 +0000 UTC m=+1615.689025462" watchObservedRunningTime="2026-09-14 08:14:50.91585362 +0000 UTC m=+1615.690351954"
Sep 14 08:14:51.490887 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:51.490850    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7864-b6d170"]
Sep 14 08:14:51.495637 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:51.495606    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7864-b6d170"]
Sep 14 08:14:51.905717 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:14:51.905680    2577 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7864-b6d170" container="test-curl-7864-b6d170"
Sep 14 08:14:52.910687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:52.910658    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7864-b6d170" podUID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" containerName="test-curl-7864-b6d170" containerID="cri-o://aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5" gracePeriod=2
Sep 14 08:14:53.086023 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.086000    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:53.164002 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.163906    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8r6ks\" (UniqueName: \"kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks\") pod \"07f33594-be11-4abc-bcfb-5be3274e429c\" (UID: \"07f33594-be11-4abc-bcfb-5be3274e429c\") "
Sep 14 08:14:53.165973 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.165950    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks" (OuterVolumeSpecName: "kube-api-access-8r6ks") pod "07f33594-be11-4abc-bcfb-5be3274e429c" (UID: "07f33594-be11-4abc-bcfb-5be3274e429c"). InnerVolumeSpecName "kube-api-access-8r6ks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:53.265315 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.265285    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8r6ks\" (UniqueName: \"kubernetes.io/projected/07f33594-be11-4abc-bcfb-5be3274e429c-kube-api-access-8r6ks\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:14:53.474456 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.474376    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:14:53.760731 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.760653    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="07f33594-be11-4abc-bcfb-5be3274e429c" path="/var/lib/kubelet/pods/07f33594-be11-4abc-bcfb-5be3274e429c/volumes"
Sep 14 08:14:53.916441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.916409    2577 generic.go:356] "Generic (PLEG): container finished" podID="07f33594-be11-4abc-bcfb-5be3274e429c" containerID="75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907" exitCode=137
Sep 14 08:14:53.916892 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.916452    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-decb0d"
Sep 14 08:14:53.916892 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.916501    2577 scope.go:117] "RemoveContainer" containerID="75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907"
Sep 14 08:14:53.924837 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.924819    2577 scope.go:117] "RemoveContainer" containerID="75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907"
Sep 14 08:14:53.925118 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:14:53.925099    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907\": container with ID starting with 75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907 not found: ID does not exist" containerID="75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907"
Sep 14 08:14:53.925187 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:53.925141    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907"} err="failed to get container status \"75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907\": rpc error: code = NotFound desc = could not find container \"75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907\": container with ID starting with 75864771641a722fb2affa3f8d697dc8870269e26a633f9e85afe8ed54a9f907 not found: ID does not exist"
Sep 14 08:14:54.413112 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:54.413080    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:14:55.088406 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.088380    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:55.185598 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.185517    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g7plt\" (UniqueName: \"kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt\") pod \"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570\" (UID: \"c4fab49e-b6ce-40a0-a6b5-c3ac601c9570\") "
Sep 14 08:14:55.187605 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.187580    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt" (OuterVolumeSpecName: "kube-api-access-g7plt") pod "c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" (UID: "c4fab49e-b6ce-40a0-a6b5-c3ac601c9570"). InnerVolumeSpecName "kube-api-access-g7plt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:14:55.286728 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.286705    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g7plt\" (UniqueName: \"kubernetes.io/projected/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570-kube-api-access-g7plt\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:14:55.760101 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.760068    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" path="/var/lib/kubelet/pods/c4fab49e-b6ce-40a0-a6b5-c3ac601c9570/volumes"
Sep 14 08:14:55.927826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.927791    2577 generic.go:356] "Generic (PLEG): container finished" podID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" containerID="aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5" exitCode=137
Sep 14 08:14:55.928037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.927849    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7864-b6d170"
Sep 14 08:14:55.928037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.927892    2577 scope.go:117] "RemoveContainer" containerID="aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5"
Sep 14 08:14:55.936282 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.936258    2577 scope.go:117] "RemoveContainer" containerID="aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5"
Sep 14 08:14:55.936560 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:14:55.936540    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5\": container with ID starting with aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5 not found: ID does not exist" containerID="aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5"
Sep 14 08:14:55.936636 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:14:55.936566    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5"} err="failed to get container status \"aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5\": rpc error: code = NotFound desc = could not find container \"aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5\": container with ID starting with aafe15e18f6efda520703a82bf15d27f66d24406958d5ca014db637a2e80aaa5 not found: ID does not exist"
Sep 14 08:15:01.394947 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.394895    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:15:01.396433 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.396407    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" containerName="test-curl-7864-b6d170"
Sep 14 08:15:01.396582 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.396571    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" containerName="test-curl-7864-b6d170"
Sep 14 08:15:01.396851 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.396837    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4fab49e-b6ce-40a0-a6b5-c3ac601c9570" containerName="test-curl-7864-b6d170"
Sep 14 08:15:01.403527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.403504    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.406200 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.406177    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-7746e4\"/\"rate-test-model-7746e4-kserve-self-signed-certs\""
Sep 14 08:15:01.407890 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.407864    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:15:01.543352 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543322    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.543352 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543355    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.543555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543377    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.543555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543459    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.543555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c47lf\" (UniqueName: \"kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.543555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.543517    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644353 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644310    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644353 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644357    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644387    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644445    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644477    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c47lf\" (UniqueName: \"kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644796 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644607    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644796 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644769    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.644910 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644816    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.645003 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.644963    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.646789 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.646766    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.647097 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.647078    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.652347 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.652325    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c47lf\" (UniqueName: \"kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf\") pod \"rate-test-model-7746e4-kserve-68f74b7447-swmhj\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") " pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.714359 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.714326    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:01.986382 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:01.986355    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:15:01.987678 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:15:01.987648    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6132095_bb00_4623_80a0_2cf05ef2754b.slice/crio-1080d19063f209fe3a7d3c8d890a84ae6c701ba99e75aecd31d37a85ddf915f9 WatchSource:0}: Error finding container 1080d19063f209fe3a7d3c8d890a84ae6c701ba99e75aecd31d37a85ddf915f9: Status 404 returned error can't find the container with id 1080d19063f209fe3a7d3c8d890a84ae6c701ba99e75aecd31d37a85ddf915f9
Sep 14 08:15:02.966037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:02.965999    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" event={"ID":"a6132095-bb00-4623-80a0-2cf05ef2754b","Type":"ContainerStarted","Data":"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"}
Sep 14 08:15:02.966450 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:02.966049    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" event={"ID":"a6132095-bb00-4623-80a0-2cf05ef2754b","Type":"ContainerStarted","Data":"1080d19063f209fe3a7d3c8d890a84ae6c701ba99e75aecd31d37a85ddf915f9"}
Sep 14 08:15:02.984950 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:02.984884    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" podStartSLOduration=1.984866948 podStartE2EDuration="1.984866948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:02.982532828 +0000 UTC m=+1627.757031163" watchObservedRunningTime="2026-09-14 08:15:02.984866948 +0000 UTC m=+1627.759365283"
Sep 14 08:15:08.493318 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.493282    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" podUID="9e19cb79-fb50-4e0c-8f74-178264461f32" containerName="main" containerID="cri-o://4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103" gracePeriod=45
Sep 14 08:15:08.748184 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.748124    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:15:08.807762 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.807735    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.807762 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.807763    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.807960 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.807813    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.807960 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.807845    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qws2t\" (UniqueName: \"kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.808050 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.807975    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.808050 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808015    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs\") pod \"9e19cb79-fb50-4e0c-8f74-178264461f32\" (UID: \"9e19cb79-fb50-4e0c-8f74-178264461f32\") "
Sep 14 08:15:08.808129 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808052    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home" (OuterVolumeSpecName: "home") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:08.808129 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808060    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:08.808234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808133    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:08.808351 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808334    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.808408 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808352    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.808408 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.808362    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.810081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.810058    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm" (OuterVolumeSpecName: "dshm") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:08.810081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.810068    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t" (OuterVolumeSpecName: "kube-api-access-qws2t") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "kube-api-access-qws2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:08.810081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.810062    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e19cb79-fb50-4e0c-8f74-178264461f32" (UID: "9e19cb79-fb50-4e0c-8f74-178264461f32"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:08.909304 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.909274    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e19cb79-fb50-4e0c-8f74-178264461f32-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.909304 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.909303    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e19cb79-fb50-4e0c-8f74-178264461f32-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.909482 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.909318    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qws2t\" (UniqueName: \"kubernetes.io/projected/9e19cb79-fb50-4e0c-8f74-178264461f32-kube-api-access-qws2t\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:08.996805 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.996769    2577 generic.go:356] "Generic (PLEG): container finished" podID="9e19cb79-fb50-4e0c-8f74-178264461f32" containerID="4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103" exitCode=0
Sep 14 08:15:08.996949 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.996816    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"
Sep 14 08:15:08.996949 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.996814    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" event={"ID":"9e19cb79-fb50-4e0c-8f74-178264461f32","Type":"ContainerDied","Data":"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"}
Sep 14 08:15:08.997047 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.996952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm" event={"ID":"9e19cb79-fb50-4e0c-8f74-178264461f32","Type":"ContainerDied","Data":"9261d82933285578030c96dacfa1cc3d15b661106ab9d56bac0f9add3d88be0b"}
Sep 14 08:15:08.997047 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:08.996983    2577 scope.go:117] "RemoveContainer" containerID="4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"
Sep 14 08:15:09.006058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.006042    2577 scope.go:117] "RemoveContainer" containerID="4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"
Sep 14 08:15:09.006311 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:15:09.006294    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103\": container with ID starting with 4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103 not found: ID does not exist" containerID="4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"
Sep 14 08:15:09.006366 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.006321    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103"} err="failed to get container status \"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103\": rpc error: code = NotFound desc = could not find container \"4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103\": container with ID starting with 4769fe1db46cf16929f2dce83e0094bbe6d0c32fde553a21e5896bb71db6a103 not found: ID does not exist"
Sep 14 08:15:09.017993 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.017968    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:15:09.024496 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.024476    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/sub-test-model-7746e4-kserve-67b7c4c5d5-z86bm"]
Sep 14 08:15:09.432204 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.432166    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" podUID="d9d75f96-daa8-476a-8a5c-03c4b787818f" containerName="main" containerID="cri-o://df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef" gracePeriod=45
Sep 14 08:15:09.683042 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.682987    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:15:09.759812 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.759783    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e19cb79-fb50-4e0c-8f74-178264461f32" path="/var/lib/kubelet/pods/9e19cb79-fb50-4e0c-8f74-178264461f32/volumes"
Sep 14 08:15:09.816499 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816465    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816499 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816510    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816551    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5wjsx\" (UniqueName: \"kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816576    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816868 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816718    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir\") pod \"d9d75f96-daa8-476a-8a5c-03c4b787818f\" (UID: \"d9d75f96-daa8-476a-8a5c-03c4b787818f\") "
Sep 14 08:15:09.816868 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816780    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home" (OuterVolumeSpecName: "home") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:09.817010 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816959    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache" (OuterVolumeSpecName: "model-cache") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:09.817010 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.816978    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:09.817145 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.817130    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:09.817188 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.817152    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:09.817188 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.817167    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:09.818785 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.818759    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm" (OuterVolumeSpecName: "dshm") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:15:09.818853 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.818830    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:15:09.818958 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.818937    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx" (OuterVolumeSpecName: "kube-api-access-5wjsx") pod "d9d75f96-daa8-476a-8a5c-03c4b787818f" (UID: "d9d75f96-daa8-476a-8a5c-03c4b787818f"). InnerVolumeSpecName "kube-api-access-5wjsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:15:09.918309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.918275    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9d75f96-daa8-476a-8a5c-03c4b787818f-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:09.918309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.918304    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5wjsx\" (UniqueName: \"kubernetes.io/projected/d9d75f96-daa8-476a-8a5c-03c4b787818f-kube-api-access-5wjsx\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:09.918309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:09.918312    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9d75f96-daa8-476a-8a5c-03c4b787818f-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:15:10.003548 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.003471    2577 generic.go:356] "Generic (PLEG): container finished" podID="d9d75f96-daa8-476a-8a5c-03c4b787818f" containerID="df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef" exitCode=0
Sep 14 08:15:10.003548 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.003516    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"
Sep 14 08:15:10.003548 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.003525    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" event={"ID":"d9d75f96-daa8-476a-8a5c-03c4b787818f","Type":"ContainerDied","Data":"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"}
Sep 14 08:15:10.003772 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.003554    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz" event={"ID":"d9d75f96-daa8-476a-8a5c-03c4b787818f","Type":"ContainerDied","Data":"360e2dc72cec2e852ecbb2cbd5bb64daeddfac4f4c0322dd4c4df10cc9f3113e"}
Sep 14 08:15:10.003772 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.003574    2577 scope.go:117] "RemoveContainer" containerID="df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"
Sep 14 08:15:10.014178 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.014156    2577 scope.go:117] "RemoveContainer" containerID="df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"
Sep 14 08:15:10.014676 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:15:10.014655    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef\": container with ID starting with df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef not found: ID does not exist" containerID="df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"
Sep 14 08:15:10.014824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.014792    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef"} err="failed to get container status \"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef\": rpc error: code = NotFound desc = could not find container \"df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef\": container with ID starting with df9b39164304c258b9963f51391526505e89c742d163dec8e42ec078eb9ecbef not found: ID does not exist"
Sep 14 08:15:10.025080 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.025056    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:15:10.028613 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:10.028591    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/sub-test-model-ea9d35-kserve-57749b6d77-mcldz"]
Sep 14 08:15:11.715196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:11.715155    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:11.715196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:11.715201    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:11.728071 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:11.728045    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:11.759995 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:11.759971    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d9d75f96-daa8-476a-8a5c-03c4b787818f" path="/var/lib/kubelet/pods/d9d75f96-daa8-476a-8a5c-03c4b787818f/volumes"
Sep 14 08:15:12.029084 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:12.029000    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:15:42.602043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602010    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602362    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d9d75f96-daa8-476a-8a5c-03c4b787818f" containerName="main"
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602373    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d9d75f96-daa8-476a-8a5c-03c4b787818f" containerName="main"
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602386    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e19cb79-fb50-4e0c-8f74-178264461f32" containerName="main"
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602392    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e19cb79-fb50-4e0c-8f74-178264461f32" containerName="main"
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602462    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="d9d75f96-daa8-476a-8a5c-03c4b787818f" containerName="main"
Sep 14 08:15:42.602568 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.602470    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e19cb79-fb50-4e0c-8f74-178264461f32" containerName="main"
Sep 14 08:15:42.605709 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.605690    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.609570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.609549    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"kube-root-ca.crt\""
Sep 14 08:15:42.609706 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.609561    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"openshift-service-ca.crt\""
Sep 14 08:15:42.609706 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.609561    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"rate-test-model-ea9d35-kserve-self-signed-certs\""
Sep 14 08:15:42.609866 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.609570    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-ea9d35\"/\"default-dockercfg-5lrdl\""
Sep 14 08:15:42.615793 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.615769    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:15:42.701691 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701661    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.701821 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701713    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bpkx\" (UniqueName: \"kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.701821 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.701937 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701818    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.701937 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701847    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.701937 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.701870    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.802601 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802569    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.802601 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802604    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.802833 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802623    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.802833 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802646    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.802833 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802774    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.803007 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.802866    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bpkx\" (UniqueName: \"kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.803058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.803022    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.803096 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.803059    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.803160 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.803131    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.804967 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.804946    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.805225 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.805208    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.812349 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.812327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bpkx\" (UniqueName: \"kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx\") pod \"rate-test-model-ea9d35-kserve-7647d9c587-wrfm2\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") " pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:42.915632 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:42.915596    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:43.186020 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:43.185997    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:15:43.188401 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:15:43.188372    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod727a98ba_c09d_4470_a089_70c092265bae.slice/crio-e97a768360376ee1c1fe17db1a0e211d9729a3ad4d778c895ecc76193310d9a4 WatchSource:0}: Error finding container e97a768360376ee1c1fe17db1a0e211d9729a3ad4d778c895ecc76193310d9a4: Status 404 returned error can't find the container with id e97a768360376ee1c1fe17db1a0e211d9729a3ad4d778c895ecc76193310d9a4
Sep 14 08:15:44.177165 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:44.177126    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" event={"ID":"727a98ba-c09d-4470-a089-70c092265bae","Type":"ContainerStarted","Data":"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"}
Sep 14 08:15:44.177543 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:44.177173    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" event={"ID":"727a98ba-c09d-4470-a089-70c092265bae","Type":"ContainerStarted","Data":"e97a768360376ee1c1fe17db1a0e211d9729a3ad4d778c895ecc76193310d9a4"}
Sep 14 08:15:44.194899 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:44.194854    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" podStartSLOduration=2.194839765 podStartE2EDuration="2.194839765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:15:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:15:44.193479762 +0000 UTC m=+1668.967978109" watchObservedRunningTime="2026-09-14 08:15:44.194839765 +0000 UTC m=+1668.969338148"
Sep 14 08:15:52.916199 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:52.916157    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:52.916199 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:52.916212    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:52.928677 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:52.928654    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:15:53.232426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:15:53.232346    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:16:21.785563 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:21.785481    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:16:22.771425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:22.771394    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:16:23.537211 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.537179    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:16:23.540981 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.540963    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.543391 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.543372    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 14 08:16:23.549337 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.543790    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-7f1623-openshift-default-dockercfg-qx4hx\""
Sep 14 08:16:23.555156 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.555128    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:16:23.574345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574319    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574469 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574350    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574469 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574440    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574558    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574584    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wpjcp\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574702 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574702 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574673    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.574702 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.574694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675564 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675535    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675573    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675593    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wpjcp\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675618    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675703 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675640    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675941 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675828    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.675941 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675901    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675959    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675988    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.675960    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676059 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.676049    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676288 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.676081    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676288 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.676180    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.676499 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.676480    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.677890 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.677868    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.677990 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.677900    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.683615 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.683590    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.683711 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.683642    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wpjcp\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp\") pod \"e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") " pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:23.859593 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:23.859500    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:24.137357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.137329    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:16:24.139254 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:16:24.139209    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63cbea1c_1a8c_4d9b_9c6e_882262d5e386.slice/crio-8f4449a959e5e8f55ee7f8b964a38c12f24119567f4c0826ff2ca7651ed82ed7 WatchSource:0}: Error finding container 8f4449a959e5e8f55ee7f8b964a38c12f24119567f4c0826ff2ca7651ed82ed7: Status 404 returned error can't find the container with id 8f4449a959e5e8f55ee7f8b964a38c12f24119567f4c0826ff2ca7651ed82ed7
Sep 14 08:16:24.141478 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.141441    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:24.141592 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.141519    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:24.141592 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.141563    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:24.370748 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.370705    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerStarted","Data":"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"}
Sep 14 08:16:24.370950 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.370775    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerStarted","Data":"8f4449a959e5e8f55ee7f8b964a38c12f24119567f4c0826ff2ca7651ed82ed7"}
Sep 14 08:16:24.390214 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.390105    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podStartSLOduration=1.39008571 podStartE2EDuration="1.39008571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:24.388083263 +0000 UTC m=+1709.162581598" watchObservedRunningTime="2026-09-14 08:16:24.39008571 +0000 UTC m=+1709.164584046"
Sep 14 08:16:24.859608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:24.859561    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:25.864686 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:25.864656    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:26.380483 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:26.380450    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:26.381283 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:26.381261    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:36.804830 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:36.804765    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" podUID="a6132095-bb00-4623-80a0-2cf05ef2754b" containerName="main" containerID="cri-o://34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6" gracePeriod=45
Sep 14 08:16:37.072432 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.072406    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:16:37.212150 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212122    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212190    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212229    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212261    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212283    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212345 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212312    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c47lf\" (UniqueName: \"kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf\") pod \"a6132095-bb00-4623-80a0-2cf05ef2754b\" (UID: \"a6132095-bb00-4623-80a0-2cf05ef2754b\") "
Sep 14 08:16:37.212653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212388    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache" (OuterVolumeSpecName: "model-cache") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:37.212653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212469    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home" (OuterVolumeSpecName: "home") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:37.212653 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212485    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:37.212819 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212660    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.212819 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212673    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.212819 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.212683    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.214339 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.214316    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm" (OuterVolumeSpecName: "dshm") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:37.214434 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.214414    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:37.214434 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.214411    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf" (OuterVolumeSpecName: "kube-api-access-c47lf") pod "a6132095-bb00-4623-80a0-2cf05ef2754b" (UID: "a6132095-bb00-4623-80a0-2cf05ef2754b"). InnerVolumeSpecName "kube-api-access-c47lf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:37.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.313691    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6132095-bb00-4623-80a0-2cf05ef2754b-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.313722    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a6132095-bb00-4623-80a0-2cf05ef2754b-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.313732    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c47lf\" (UniqueName: \"kubernetes.io/projected/a6132095-bb00-4623-80a0-2cf05ef2754b-kube-api-access-c47lf\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:37.439005 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.438975    2577 generic.go:356] "Generic (PLEG): container finished" podID="a6132095-bb00-4623-80a0-2cf05ef2754b" containerID="34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6" exitCode=0
Sep 14 08:16:37.439181 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.439019    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"
Sep 14 08:16:37.439181 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.439065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" event={"ID":"a6132095-bb00-4623-80a0-2cf05ef2754b","Type":"ContainerDied","Data":"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"}
Sep 14 08:16:37.439181 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.439112    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj" event={"ID":"a6132095-bb00-4623-80a0-2cf05ef2754b","Type":"ContainerDied","Data":"1080d19063f209fe3a7d3c8d890a84ae6c701ba99e75aecd31d37a85ddf915f9"}
Sep 14 08:16:37.439181 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.439135    2577 scope.go:117] "RemoveContainer" containerID="34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"
Sep 14 08:16:37.448417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.448400    2577 scope.go:117] "RemoveContainer" containerID="34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"
Sep 14 08:16:37.448716 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:16:37.448699    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6\": container with ID starting with 34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6 not found: ID does not exist" containerID="34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"
Sep 14 08:16:37.448787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.448730    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6"} err="failed to get container status \"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6\": rpc error: code = NotFound desc = could not find container \"34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6\": container with ID starting with 34e6c8dda6fc6b41df36a71922ee8b2d040822f5970eed05f302f7ed144aa7f6 not found: ID does not exist"
Sep 14 08:16:37.459879 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.459839    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:16:37.463278 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.463254    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-7746e4/rate-test-model-7746e4-kserve-68f74b7447-swmhj"]
Sep 14 08:16:37.760605 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.760575    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6132095-bb00-4623-80a0-2cf05ef2754b" path="/var/lib/kubelet/pods/a6132095-bb00-4623-80a0-2cf05ef2754b/volumes"
Sep 14 08:16:37.800414 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:37.800377    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" podUID="727a98ba-c09d-4470-a089-70c092265bae" containerName="main" containerID="cri-o://30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f" gracePeriod=45
Sep 14 08:16:38.058316 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.058292    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:16:38.222810 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.222774    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.222810 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.222811    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.223115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.222861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bpkx\" (UniqueName: \"kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.223115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.222902    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.223115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.222948    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.223115 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home\") pod \"727a98ba-c09d-4470-a089-70c092265bae\" (UID: \"727a98ba-c09d-4470-a089-70c092265bae\") "
Sep 14 08:16:38.223308 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223105    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache" (OuterVolumeSpecName: "model-cache") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:38.223308 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223128    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:38.223308 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223266    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home" (OuterVolumeSpecName: "home") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:38.223467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223404    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.223467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223423    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.223467 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.223437    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.225146 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.225122    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm" (OuterVolumeSpecName: "dshm") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:16:38.225146 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.225130    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:16:38.225285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.225171    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx" (OuterVolumeSpecName: "kube-api-access-5bpkx") pod "727a98ba-c09d-4470-a089-70c092265bae" (UID: "727a98ba-c09d-4470-a089-70c092265bae"). InnerVolumeSpecName "kube-api-access-5bpkx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:16:38.323998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.323881    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bpkx\" (UniqueName: \"kubernetes.io/projected/727a98ba-c09d-4470-a089-70c092265bae-kube-api-access-5bpkx\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.323998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.323937    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/727a98ba-c09d-4470-a089-70c092265bae-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.323998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.323952    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/727a98ba-c09d-4470-a089-70c092265bae-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:16:38.445968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.445909    2577 generic.go:356] "Generic (PLEG): container finished" podID="727a98ba-c09d-4470-a089-70c092265bae" containerID="30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f" exitCode=0
Sep 14 08:16:38.445968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.445965    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"
Sep 14 08:16:38.446195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.446021    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" event={"ID":"727a98ba-c09d-4470-a089-70c092265bae","Type":"ContainerDied","Data":"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"}
Sep 14 08:16:38.446195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.446065    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2" event={"ID":"727a98ba-c09d-4470-a089-70c092265bae","Type":"ContainerDied","Data":"e97a768360376ee1c1fe17db1a0e211d9729a3ad4d778c895ecc76193310d9a4"}
Sep 14 08:16:38.446195 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.446091    2577 scope.go:117] "RemoveContainer" containerID="30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"
Sep 14 08:16:38.457209 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.457182    2577 scope.go:117] "RemoveContainer" containerID="30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"
Sep 14 08:16:38.457593 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:16:38.457571    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f\": container with ID starting with 30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f not found: ID does not exist" containerID="30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"
Sep 14 08:16:38.457810 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.457783    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f"} err="failed to get container status \"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f\": rpc error: code = NotFound desc = could not find container \"30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f\": container with ID starting with 30e3d8e31ba2991860102a560e65bbdbe29079ba7954f977c57c9a801a9f022f not found: ID does not exist"
Sep 14 08:16:38.469177 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.469149    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:16:38.470803 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:38.470785    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-ea9d35/rate-test-model-ea9d35-kserve-7647d9c587-wrfm2"]
Sep 14 08:16:39.760325 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:39.760293    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="727a98ba-c09d-4470-a089-70c092265bae" path="/var/lib/kubelet/pods/727a98ba-c09d-4470-a089-70c092265bae/volumes"
Sep 14 08:16:42.423110 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:42.423066    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 14 08:16:42.423493 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:42.423149    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 14 08:16:44.478803 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.478766    2577 generic.go:356] "Generic (PLEG): container finished" podID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerID="5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7" exitCode=0
Sep 14 08:16:44.479360 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.478843    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerDied","Data":"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"}
Sep 14 08:16:44.479448 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.479359    2577 scope.go:117] "RemoveContainer" containerID="5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"
Sep 14 08:16:44.480612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.480575    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:44.480684 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.480643    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:44.480684 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:44.480676    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:45.488833 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:45.488787    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerStarted","Data":"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"}
Sep 14 08:16:45.860708 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:45.860620    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:45.860860 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:45.860788    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 14 08:16:45.860860 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:45.860838    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 14 08:16:46.859785 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:46.859747    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 14 08:16:46.860177 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:46.859799    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 14 08:16:47.860799 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:47.860759    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 14 08:16:47.861323 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:47.860840    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 14 08:16:48.860753 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:48.860714    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 14 08:16:48.860966 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:48.860774    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 14 08:16:49.860734 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:49.860701    2577 patch_prober.go:28] interesting pod/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 14 08:16:49.860936 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:49.860773    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 14 08:16:50.863788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:50.863757    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:50.864180 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:50.864118    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:50.864726 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:50.864707    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:16:52.298682 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.298653    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:16:52.299150 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299133    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="727a98ba-c09d-4470-a089-70c092265bae" containerName="main"
Sep 14 08:16:52.299197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299153    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="727a98ba-c09d-4470-a089-70c092265bae" containerName="main"
Sep 14 08:16:52.299197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299175    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6132095-bb00-4623-80a0-2cf05ef2754b" containerName="main"
Sep 14 08:16:52.299197 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299181    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6132095-bb00-4623-80a0-2cf05ef2754b" containerName="main"
Sep 14 08:16:52.299295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299247    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6132095-bb00-4623-80a0-2cf05ef2754b" containerName="main"
Sep 14 08:16:52.299295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.299256    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="727a98ba-c09d-4470-a089-70c092265bae" containerName="main"
Sep 14 08:16:52.301355 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.301336    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.303897 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.303877    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-56708c-openshift-default-dockercfg-65tz9\""
Sep 14 08:16:52.314424 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.314404    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:16:52.354618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.354811 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354645    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.354811 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354747    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.354811 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354785    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.355028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354860    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.355028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354939    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.355028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354965    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9nlp\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.355028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.354991    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.355028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.355025    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455453 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455421    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455467    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455500    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455518    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455570    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455610    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455636    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9nlp\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.455690 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455663    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455697    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455867    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.455941    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456082 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.456040    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456228 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.456195    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.456354 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.456334    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.457925 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.457896    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.458183 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.458166    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.464108 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.464085    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.464280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.464257    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9nlp\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp\") pod \"e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") " pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.612166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.612072    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:52.886859 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.886751    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:16:52.889348 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:16:52.889321    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70782bda_5bda_417a_8d1c_a9c06f005c7c.slice/crio-bb30e8d0258c1c3852b0ebc4770b237ccc9f8b0332a07458c0727a0bcc862c58 WatchSource:0}: Error finding container bb30e8d0258c1c3852b0ebc4770b237ccc9f8b0332a07458c0727a0bcc862c58: Status 404 returned error can't find the container with id bb30e8d0258c1c3852b0ebc4770b237ccc9f8b0332a07458c0727a0bcc862c58
Sep 14 08:16:52.891549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.891515    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:52.891634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.891585    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:52.891634 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:52.891616    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:16:53.535323 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:53.535291    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerStarted","Data":"d4833087c62bd1906ce3b5a9879b0bd2f34df2e7ffe98c53e655765e91d38ced"}
Sep 14 08:16:53.535685 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:53.535348    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerStarted","Data":"bb30e8d0258c1c3852b0ebc4770b237ccc9f8b0332a07458c0727a0bcc862c58"}
Sep 14 08:16:53.555303 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:53.555257    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podStartSLOduration=1.5552409649999999 podStartE2EDuration="1.555240965s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:53.554470529 +0000 UTC m=+1738.328968878" watchObservedRunningTime="2026-09-14 08:16:53.555240965 +0000 UTC m=+1738.329739300"
Sep 14 08:16:53.612986 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:53.612952    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:53.617869 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:53.617841    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:54.540471 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:54.540438    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:54.541472 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:54.541454    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:16:58.838488 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.838454    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:16:58.841028 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.841001    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:58.843691 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.843670    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-56708c\""
Sep 14 08:16:58.843815 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.843714    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-56708c-dockercfg-pkswf\""
Sep 14 08:16:58.852498 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.852473    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:16:58.858559 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.858536    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:16:58.861194 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.861176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:58.872441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.872418    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:16:58.906084 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.906053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9bqs6\" (UniqueName: \"kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:58.906252 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.906108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:58.906252 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.906175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jl2v\" (UniqueName: \"kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:58.906252 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:58.906206    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.007314 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.007273    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9bqs6\" (UniqueName: \"kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.007488 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.007334    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.007488 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.007362    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jl2v\" (UniqueName: \"kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.007488 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.007379    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.008138 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.008119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.008138 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.008127    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.015393 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.015363    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jl2v\" (UniqueName: \"kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v\") pod \"payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.015543 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.015481    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9bqs6\" (UniqueName: \"kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6\") pod \"payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.151203 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.151176    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.172858 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.172825    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.427802 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.427777    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:16:59.430309 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:16:59.430280    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d415092_bc78_4b37_812d_28322f71eb70.slice/crio-c611912b5e35b0c60befe9b7ae4af4d9121c6c792e877579a1188c0e8fcb7838 WatchSource:0}: Error finding container c611912b5e35b0c60befe9b7ae4af4d9121c6c792e877579a1188c0e8fcb7838: Status 404 returned error can't find the container with id c611912b5e35b0c60befe9b7ae4af4d9121c6c792e877579a1188c0e8fcb7838
Sep 14 08:16:59.457397 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.457372    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:16:59.469006 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:16:59.468983    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8f4805_505f_4627_94d3_39a219961e19.slice/crio-fe99861c747e7a117fd9aeba5639c2fb2a30bc55644c6769cd891a8ff07358bf WatchSource:0}: Error finding container fe99861c747e7a117fd9aeba5639c2fb2a30bc55644c6769cd891a8ff07358bf: Status 404 returned error can't find the container with id fe99861c747e7a117fd9aeba5639c2fb2a30bc55644c6769cd891a8ff07358bf
Sep 14 08:16:59.566456 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.566411    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" event={"ID":"4d415092-bc78-4b37-812d-28322f71eb70","Type":"ContainerStarted","Data":"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"}
Sep 14 08:16:59.566648 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.566488    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" event={"ID":"4d415092-bc78-4b37-812d-28322f71eb70","Type":"ContainerStarted","Data":"c611912b5e35b0c60befe9b7ae4af4d9121c6c792e877579a1188c0e8fcb7838"}
Sep 14 08:16:59.566648 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.566522    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:16:59.568138 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.568115    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" event={"ID":"9b8f4805-505f-4627-94d3-39a219961e19","Type":"ContainerStarted","Data":"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"}
Sep 14 08:16:59.568258 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.568159    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:16:59.568258 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.568175    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" event={"ID":"9b8f4805-505f-4627-94d3-39a219961e19","Type":"ContainerStarted","Data":"fe99861c747e7a117fd9aeba5639c2fb2a30bc55644c6769cd891a8ff07358bf"}
Sep 14 08:16:59.585104 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.585062    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" podStartSLOduration=1.585044104 podStartE2EDuration="1.585044104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:59.583230884 +0000 UTC m=+1744.357729221" watchObservedRunningTime="2026-09-14 08:16:59.585044104 +0000 UTC m=+1744.359542438"
Sep 14 08:16:59.601268 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:16:59.601226    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" podStartSLOduration=1.601211772 podStartE2EDuration="1.601211772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:16:59.600052068 +0000 UTC m=+1744.374550404" watchObservedRunningTime="2026-09-14 08:16:59.601211772 +0000 UTC m=+1744.375710107"
Sep 14 08:17:10.575319 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:10.575239    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:17:10.575808 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:10.575322    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:17:13.640411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.640321    2577 generic.go:356] "Generic (PLEG): container finished" podID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerID="d4833087c62bd1906ce3b5a9879b0bd2f34df2e7ffe98c53e655765e91d38ced" exitCode=0
Sep 14 08:17:13.640411 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.640394    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerDied","Data":"d4833087c62bd1906ce3b5a9879b0bd2f34df2e7ffe98c53e655765e91d38ced"}
Sep 14 08:17:13.640875 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.640857    2577 scope.go:117] "RemoveContainer" containerID="d4833087c62bd1906ce3b5a9879b0bd2f34df2e7ffe98c53e655765e91d38ced"
Sep 14 08:17:13.641976 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.641942    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:17:13.642041 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.642016    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:17:13.642081 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:13.642051    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:17:14.647007 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:14.646967    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerStarted","Data":"17d6c64fa6184a324762de44a6a864bd76860e2a9212b4b3181d025158bca727"}
Sep 14 08:17:15.612249 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:15.612217    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:17:15.612425 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:15.612395    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 14 08:17:15.612477 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:15.612453    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 14 08:17:16.613169 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:16.613136    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 14 08:17:16.613574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:16.613190    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 14 08:17:17.612446 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:17.612407    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 14 08:17:17.612625 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:17.612468    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 14 08:17:18.612604 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:18.612565    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused" start-of-body=
Sep 14 08:17:18.613032 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:18.612623    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.86:15021/healthz/ready\": dial tcp 10.132.0.86:15021: connect: connection refused"
Sep 14 08:17:19.764053 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:19.764021    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:17:19.764531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:19.764395    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:17:19.765305 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:19.765287    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:17:25.785027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:25.784995    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2_9b8f4805-505f-4627-94d3-39a219961e19/payload-processing/0.log"
Sep 14 08:17:55.825751 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:55.825723    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:17:55.831581 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:17:55.831558    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:18:06.939531 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:06.939501    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2_9b8f4805-505f-4627-94d3-39a219961e19/payload-processing/0.log"
Sep 14 08:18:07.475588 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.475553    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:18:07.480875 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.478538    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.482163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.481871    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-925f2a-openshift-default-dockercfg-svwfd\""
Sep 14 08:18:07.496091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.496066    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:18:07.648064 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648087    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648181    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648234 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648256    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648311    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7szz\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648357    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.648398 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.648390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.748851 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.748770    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.748851 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.748837    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7szz\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.748869    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.748908    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.748969    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749002    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749023    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749068    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749103    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749214    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749327    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749400 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749368    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749484    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.749757 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.749738    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.751198 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.751178    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.751394 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.751377    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.756842 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.756820    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.756986 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.756902    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7szz\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz\") pod \"e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") " pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:07.792922 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:07.792900    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:08.067714 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.067692    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:18:08.070072 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:18:08.070046    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c64a43f_aa33_4982_954d_ea4270310e83.slice/crio-57b7023640b75946cdad65971e79b95e3d821be58996e13306293333d33f5a0b WatchSource:0}: Error finding container 57b7023640b75946cdad65971e79b95e3d821be58996e13306293333d33f5a0b: Status 404 returned error can't find the container with id 57b7023640b75946cdad65971e79b95e3d821be58996e13306293333d33f5a0b
Sep 14 08:18:08.072428 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.072397    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:08.072495 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.072463    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:08.072495 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.072490    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:08.924258 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.924224    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerStarted","Data":"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"}
Sep 14 08:18:08.924447 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.924279    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerStarted","Data":"57b7023640b75946cdad65971e79b95e3d821be58996e13306293333d33f5a0b"}
Sep 14 08:18:08.945272 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:08.945226    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podStartSLOduration=1.9452113290000002 podStartE2EDuration="1.945211329s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:18:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:18:08.943625851 +0000 UTC m=+1813.718124211" watchObservedRunningTime="2026-09-14 08:18:08.945211329 +0000 UTC m=+1813.719709664"
Sep 14 08:18:09.793476 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:09.793446    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:09.799441 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:09.799418    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:09.928856 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:09.928830    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:09.929692 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:09.929670    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:32.052296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.052243    2577 generic.go:356] "Generic (PLEG): container finished" podID="7c64a43f-aa33-4982-954d-ea4270310e83" containerID="93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a" exitCode=0
Sep 14 08:18:32.052799 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.052355    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerDied","Data":"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"}
Sep 14 08:18:32.053071 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.053046    2577 scope.go:117] "RemoveContainer" containerID="93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"
Sep 14 08:18:32.060066 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.060032    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:32.060196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.060102    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:32.060196 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:32.060132    2577 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236200Ki","pods":"250"}
Sep 14 08:18:33.059666 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:33.059632    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerStarted","Data":"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"}
Sep 14 08:18:33.794027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:33.793984    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:33.794393 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:33.794367    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 14 08:18:33.794558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:33.794535    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 14 08:18:34.793497 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:34.793460    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 14 08:18:34.794102 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:34.794069    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 14 08:18:35.794064 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:35.794024    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 14 08:18:35.794563 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:35.794088    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 14 08:18:36.793436 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:36.793395    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 14 08:18:36.793717 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:36.793687    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 14 08:18:37.794380 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:37.794332    2577 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body=
Sep 14 08:18:37.794903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:37.794421    2577 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused"
Sep 14 08:18:38.953128 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:38.953069    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:38.953615 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:38.953506    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:38.954550 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:38.954527    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:18:59.760875 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:59.760844    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:18:59.761437 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:18:59.761084    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy" containerID="cri-o://bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e" gracePeriod=30
Sep 14 08:19:05.009606 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.009583    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:19:05.094139 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094067    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094139 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094108    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094139 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094132    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t7szz\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094166    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094200    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094279    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094343    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094377    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094426 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094408    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo\") pod \"7c64a43f-aa33-4982-954d-ea4270310e83\" (UID: \"7c64a43f-aa33-4982-954d-ea4270310e83\") "
Sep 14 08:19:05.094723 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:19:05.094723 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094529    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:05.094723 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094654    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data" (OuterVolumeSpecName: "istio-data") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:05.094873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094722    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:05.094873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094761    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:05.094873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094833    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.094873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094853    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7c64a43f-aa33-4982-954d-ea4270310e83-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.094873 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.094866    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.096445 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.096422    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token" (OuterVolumeSpecName: "istio-token") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:05.096541 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.096464    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:19:05.096541 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.096501    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:19:05.096615 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.096540    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz" (OuterVolumeSpecName: "kube-api-access-t7szz") pod "7c64a43f-aa33-4982-954d-ea4270310e83" (UID: "7c64a43f-aa33-4982-954d-ea4270310e83"). InnerVolumeSpecName "kube-api-access-t7szz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:19:05.195326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195297    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.195326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195320    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.195326 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195329    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7c64a43f-aa33-4982-954d-ea4270310e83-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.195527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195339    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7c64a43f-aa33-4982-954d-ea4270310e83-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.195527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195347    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t7szz\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-kube-api-access-t7szz\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.195527 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.195355    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7c64a43f-aa33-4982-954d-ea4270310e83-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:19:05.228890 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.228857    2577 generic.go:356] "Generic (PLEG): container finished" podID="7c64a43f-aa33-4982-954d-ea4270310e83" containerID="bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e" exitCode=0
Sep 14 08:19:05.229025 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.228929    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"
Sep 14 08:19:05.229025 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.228952    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerDied","Data":"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"}
Sep 14 08:19:05.229025 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.229006    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c" event={"ID":"7c64a43f-aa33-4982-954d-ea4270310e83","Type":"ContainerDied","Data":"57b7023640b75946cdad65971e79b95e3d821be58996e13306293333d33f5a0b"}
Sep 14 08:19:05.229133 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.229028    2577 scope.go:117] "RemoveContainer" containerID="bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"
Sep 14 08:19:05.238705 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.238686    2577 scope.go:117] "RemoveContainer" containerID="93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"
Sep 14 08:19:05.246558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.246543    2577 scope.go:117] "RemoveContainer" containerID="bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"
Sep 14 08:19:05.246808 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:19:05.246792    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e\": container with ID starting with bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e not found: ID does not exist" containerID="bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"
Sep 14 08:19:05.246857 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.246814    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e"} err="failed to get container status \"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e\": rpc error: code = NotFound desc = could not find container \"bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e\": container with ID starting with bb6e6fd8619ee7666ccc407072feb76193336215e1b05fbb317ea35dcb5fac9e not found: ID does not exist"
Sep 14 08:19:05.246857 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.246828    2577 scope.go:117] "RemoveContainer" containerID="93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"
Sep 14 08:19:05.247087 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:19:05.247068    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a\": container with ID starting with 93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a not found: ID does not exist" containerID="93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"
Sep 14 08:19:05.247130 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.247095    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a"} err="failed to get container status \"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a\": rpc error: code = NotFound desc = could not find container \"93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a\": container with ID starting with 93ee1e08f0f73c2909b9f799a2862ba01cb01b5b5fdb25016dd771281cf5eb0a not found: ID does not exist"
Sep 14 08:19:05.252516 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.252494    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:19:05.256372 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.256353    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-925f2a-openshift-default-676cf7b494-f7d5c"]
Sep 14 08:19:05.762048 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:19:05.762019    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" path="/var/lib/kubelet/pods/7c64a43f-aa33-4982-954d-ea4270310e83/volumes"
Sep 14 08:20:12.987693 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:12.987655    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:20:12.988273 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:12.987856    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy" containerID="cri-o://29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad" gracePeriod=30
Sep 14 08:20:18.244907 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.244880    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:20:18.362001 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.361870    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362001 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.361942    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362001 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.361982    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wpjcp\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362009    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362039    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362079    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362104    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362142    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362280 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362204    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs\") pod \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\" (UID: \"63cbea1c-1a8c-4d9b-9c6e-882262d5e386\") "
Sep 14 08:20:18.362590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362309    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:18.362590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362372    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data" (OuterVolumeSpecName: "istio-data") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:18.362590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362510    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:18.362590 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362523    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:18.362787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362607    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:18.362787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362619    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.362787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362639    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.362787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362652    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.362787 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.362665    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.364420 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.364391    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token" (OuterVolumeSpecName: "istio-token") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:18.364524 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.364440    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:20:18.364524 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.364490    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp" (OuterVolumeSpecName: "kube-api-access-wpjcp") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "kube-api-access-wpjcp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:18.364608 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.364512    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "63cbea1c-1a8c-4d9b-9c6e-882262d5e386" (UID: "63cbea1c-1a8c-4d9b-9c6e-882262d5e386"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:18.464070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.464028    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.464070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.464065    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.464070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.464075    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.464070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.464084    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wpjcp\" (UniqueName: \"kubernetes.io/projected/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-kube-api-access-wpjcp\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.464330 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.464093    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/63cbea1c-1a8c-4d9b-9c6e-882262d5e386-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:18.601687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.601653    2577 generic.go:356] "Generic (PLEG): container finished" podID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerID="29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad" exitCode=0
Sep 14 08:20:18.601998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.601697    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"
Sep 14 08:20:18.601998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.601734    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerDied","Data":"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"}
Sep 14 08:20:18.601998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.601788    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf" event={"ID":"63cbea1c-1a8c-4d9b-9c6e-882262d5e386","Type":"ContainerDied","Data":"8f4449a959e5e8f55ee7f8b964a38c12f24119567f4c0826ff2ca7651ed82ed7"}
Sep 14 08:20:18.601998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.601818    2577 scope.go:117] "RemoveContainer" containerID="29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"
Sep 14 08:20:18.611244 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.611225    2577 scope.go:117] "RemoveContainer" containerID="5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"
Sep 14 08:20:18.619083 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.619064    2577 scope.go:117] "RemoveContainer" containerID="29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"
Sep 14 08:20:18.619352 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:18.619334    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad\": container with ID starting with 29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad not found: ID does not exist" containerID="29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"
Sep 14 08:20:18.619401 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.619361    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad"} err="failed to get container status \"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad\": rpc error: code = NotFound desc = could not find container \"29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad\": container with ID starting with 29235ca9f6a7fe160af0330824a8c74877c5c8ed948c1146404b6295d7dcbcad not found: ID does not exist"
Sep 14 08:20:18.619401 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.619375    2577 scope.go:117] "RemoveContainer" containerID="5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"
Sep 14 08:20:18.619610 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:18.619592    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7\": container with ID starting with 5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7 not found: ID does not exist" containerID="5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"
Sep 14 08:20:18.619651 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.619618    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7"} err="failed to get container status \"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7\": rpc error: code = NotFound desc = could not find container \"5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7\": container with ID starting with 5e79197e95b3df289461c727b157486a61614bed370324bfa3ab13d6d0dedba7 not found: ID does not exist"
Sep 14 08:20:18.623330 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.623310    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:20:18.627071 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:18.627052    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-7f1623-openshift-default-6dbd5bf784-kh6nf"]
Sep 14 08:20:19.760318 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:19.760279    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" path="/var/lib/kubelet/pods/63cbea1c-1a8c-4d9b-9c6e-882262d5e386/volumes"
Sep 14 08:20:23.912200 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:23.912166    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:20:23.912619 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:23.912407    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" podUID="9b8f4805-505f-4627-94d3-39a219961e19" containerName="payload-processing" containerID="cri-o://fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b" gracePeriod=30
Sep 14 08:20:23.918575 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:23.918535    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:20:23.918765 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:23.918740    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" podUID="4d415092-bc78-4b37-812d-28322f71eb70" containerName="payload-pre-processing" containerID="cri-o://d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be" gracePeriod=30
Sep 14 08:20:24.176926 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.176850    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:20:24.203685 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.203660    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:20:24.313581 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.313553    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9bqs6\" (UniqueName: \"kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6\") pod \"4d415092-bc78-4b37-812d-28322f71eb70\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") "
Sep 14 08:20:24.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.313626    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume\") pod \"4d415092-bc78-4b37-812d-28322f71eb70\" (UID: \"4d415092-bc78-4b37-812d-28322f71eb70\") "
Sep 14 08:20:24.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.313662    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7jl2v\" (UniqueName: \"kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v\") pod \"9b8f4805-505f-4627-94d3-39a219961e19\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") "
Sep 14 08:20:24.313777 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.313701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume\") pod \"9b8f4805-505f-4627-94d3-39a219961e19\" (UID: \"9b8f4805-505f-4627-94d3-39a219961e19\") "
Sep 14 08:20:24.314123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.314090    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4d415092-bc78-4b37-812d-28322f71eb70" (UID: "4d415092-bc78-4b37-812d-28322f71eb70"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:24.314123 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.314111    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b8f4805-505f-4627-94d3-39a219961e19" (UID: "9b8f4805-505f-4627-94d3-39a219961e19"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:24.315801 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.315778    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v" (OuterVolumeSpecName: "kube-api-access-7jl2v") pod "9b8f4805-505f-4627-94d3-39a219961e19" (UID: "9b8f4805-505f-4627-94d3-39a219961e19"). InnerVolumeSpecName "kube-api-access-7jl2v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:24.315867 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.315798    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6" (OuterVolumeSpecName: "kube-api-access-9bqs6") pod "4d415092-bc78-4b37-812d-28322f71eb70" (UID: "4d415092-bc78-4b37-812d-28322f71eb70"). InnerVolumeSpecName "kube-api-access-9bqs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:24.414794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.414751    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7jl2v\" (UniqueName: \"kubernetes.io/projected/9b8f4805-505f-4627-94d3-39a219961e19-kube-api-access-7jl2v\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.414794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.414787    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8f4805-505f-4627-94d3-39a219961e19-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.414794 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.414797    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9bqs6\" (UniqueName: \"kubernetes.io/projected/4d415092-bc78-4b37-812d-28322f71eb70-kube-api-access-9bqs6\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.415068 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.414806    2577 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4d415092-bc78-4b37-812d-28322f71eb70-plugins-config-volume\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:24.631960 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.631870    2577 generic.go:356] "Generic (PLEG): container finished" podID="9b8f4805-505f-4627-94d3-39a219961e19" containerID="fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b" exitCode=0
Sep 14 08:20:24.631960 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.631928    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"
Sep 14 08:20:24.632166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.631959    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" event={"ID":"9b8f4805-505f-4627-94d3-39a219961e19","Type":"ContainerDied","Data":"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"}
Sep 14 08:20:24.632166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.632014    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2" event={"ID":"9b8f4805-505f-4627-94d3-39a219961e19","Type":"ContainerDied","Data":"fe99861c747e7a117fd9aeba5639c2fb2a30bc55644c6769cd891a8ff07358bf"}
Sep 14 08:20:24.632166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.632045    2577 scope.go:117] "RemoveContainer" containerID="fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"
Sep 14 08:20:24.633379 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.633361    2577 generic.go:356] "Generic (PLEG): container finished" podID="4d415092-bc78-4b37-812d-28322f71eb70" containerID="d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be" exitCode=0
Sep 14 08:20:24.633507 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.633391    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" event={"ID":"4d415092-bc78-4b37-812d-28322f71eb70","Type":"ContainerDied","Data":"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"}
Sep 14 08:20:24.633507 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.633417    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"
Sep 14 08:20:24.633507 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.633452    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2" event={"ID":"4d415092-bc78-4b37-812d-28322f71eb70","Type":"ContainerDied","Data":"c611912b5e35b0c60befe9b7ae4af4d9121c6c792e877579a1188c0e8fcb7838"}
Sep 14 08:20:24.640649 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.640630    2577 scope.go:117] "RemoveContainer" containerID="fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"
Sep 14 08:20:24.640946 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:24.640900    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b\": container with ID starting with fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b not found: ID does not exist" containerID="fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"
Sep 14 08:20:24.641039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.640958    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b"} err="failed to get container status \"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b\": rpc error: code = NotFound desc = could not find container \"fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b\": container with ID starting with fca198490b222c43a030b7702a138e16ef72fe125f413d53475640589844161b not found: ID does not exist"
Sep 14 08:20:24.641039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.640981    2577 scope.go:117] "RemoveContainer" containerID="d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"
Sep 14 08:20:24.649193 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.649176    2577 scope.go:117] "RemoveContainer" containerID="d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"
Sep 14 08:20:24.649469 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:24.649446    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be\": container with ID starting with d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be not found: ID does not exist" containerID="d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"
Sep 14 08:20:24.649514 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.649475    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be"} err="failed to get container status \"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be\": rpc error: code = NotFound desc = could not find container \"d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be\": container with ID starting with d6bfd285d56170b1c1132e5e0377feb1ba32c75a5472bc8eceaafa1fa7db41be not found: ID does not exist"
Sep 14 08:20:24.657555 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.657534    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:20:24.659748 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.659730    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-56708c-6bf5fc846f-vx4s2"]
Sep 14 08:20:24.679463 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.673544    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:20:24.682570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:24.682549    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-56708c-665fc54454-nqps2"]
Sep 14 08:20:25.760296 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:25.760261    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d415092-bc78-4b37-812d-28322f71eb70" path="/var/lib/kubelet/pods/4d415092-bc78-4b37-812d-28322f71eb70/volumes"
Sep 14 08:20:25.760670 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:25.760625    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b8f4805-505f-4627-94d3-39a219961e19" path="/var/lib/kubelet/pods/9b8f4805-505f-4627-94d3-39a219961e19/volumes"
Sep 14 08:20:30.522039 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:30.522001    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:20:30.522470 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:30.522216    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" containerID="cri-o://17d6c64fa6184a324762de44a6a864bd76860e2a9212b4b3181d025158bca727" gracePeriod=30
Sep 14 08:20:34.767408 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:34.767366    2577 patch_prober.go:28] interesting pod/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 14 08:20:34.767788 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:34.767430    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 14 08:20:35.693128 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.693098    2577 generic.go:356] "Generic (PLEG): container finished" podID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerID="17d6c64fa6184a324762de44a6a864bd76860e2a9212b4b3181d025158bca727" exitCode=0
Sep 14 08:20:35.693246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.693128    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerDied","Data":"17d6c64fa6184a324762de44a6a864bd76860e2a9212b4b3181d025158bca727"}
Sep 14 08:20:35.693246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.693203    2577 scope.go:117] "RemoveContainer" containerID="d4833087c62bd1906ce3b5a9879b0bd2f34df2e7ffe98c53e655765e91d38ced"
Sep 14 08:20:35.767201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.767176    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:20:35.925576 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925548    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925583    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925610    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925631    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t9nlp\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925663    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925701    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925725    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925842    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925852    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.925936    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs\") pod \"70782bda-5bda-417a-8d1c-a9c06f005c7c\" (UID: \"70782bda-5bda-417a-8d1c-a9c06f005c7c\") "
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926056    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926075    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data" (OuterVolumeSpecName: "istio-data") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926256    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926428    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926467    2577 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926487    2577 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-credential-socket\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926500    2577 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-data\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:35.926570 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.926512    2577 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-workload-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:35.928058 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.928026    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token" (OuterVolumeSpecName: "istio-token") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:35.928207 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.928188    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 14 08:20:35.928367 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.928341    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:20:35.928367 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:35.928350    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp" (OuterVolumeSpecName: "kube-api-access-t9nlp") pod "70782bda-5bda-417a-8d1c-a9c06f005c7c" (UID: "70782bda-5bda-417a-8d1c-a9c06f005c7c"). InnerVolumeSpecName "kube-api-access-t9nlp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:36.026901 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.026867    2577 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-envoy\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:36.026901 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.026895    2577 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-token\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:36.026901 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.026904    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t9nlp\" (UniqueName: \"kubernetes.io/projected/70782bda-5bda-417a-8d1c-a9c06f005c7c-kube-api-access-t9nlp\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:36.027149 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.026926    2577 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/70782bda-5bda-417a-8d1c-a9c06f005c7c-istiod-ca-cert\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:36.027149 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.026935    2577 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/70782bda-5bda-417a-8d1c-a9c06f005c7c-istio-podinfo\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:36.699898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.699864    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw" event={"ID":"70782bda-5bda-417a-8d1c-a9c06f005c7c","Type":"ContainerDied","Data":"bb30e8d0258c1c3852b0ebc4770b237ccc9f8b0332a07458c0727a0bcc862c58"}
Sep 14 08:20:36.699898 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.699885    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"
Sep 14 08:20:36.700162 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.699940    2577 scope.go:117] "RemoveContainer" containerID="17d6c64fa6184a324762de44a6a864bd76860e2a9212b4b3181d025158bca727"
Sep 14 08:20:36.722054 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.722029    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:20:36.726189 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:36.726163    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-56708c-openshift-default-967cf874-ct7sw"]
Sep 14 08:20:37.760459 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:37.760427    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" path="/var/lib/kubelet/pods/70782bda-5bda-417a-8d1c-a9c06f005c7c/volumes"
Sep 14 08:20:41.412419 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.412377    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:20:41.412908 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.412632    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" podUID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" containerName="maas-api" containerID="cri-o://9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010" gracePeriod=30
Sep 14 08:20:41.656403 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.656377    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:20:41.668813 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.668756    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpqdr\" (UniqueName: \"kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr\") pod \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") "
Sep 14 08:20:41.668951 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.668829    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") pod \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") "
Sep 14 08:20:41.668951 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.668861    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs\") pod \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") "
Sep 14 08:20:41.669073 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.668954    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca\") pod \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\" (UID: \"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa\") "
Sep 14 08:20:41.669377 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.669351    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" (UID: "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:41.671078 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.671053    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" (UID: "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:41.671337 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.671318    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" (UID: "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:41.671454 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.671433    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr" (OuterVolumeSpecName: "kube-api-access-lpqdr") pod "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" (UID: "95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa"). InnerVolumeSpecName "kube-api-access-lpqdr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:41.725902 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.725870    2577 generic.go:356] "Generic (PLEG): container finished" podID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" containerID="9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010" exitCode=0
Sep 14 08:20:41.726060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.725908    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"
Sep 14 08:20:41.726060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.725962    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" event={"ID":"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa","Type":"ContainerDied","Data":"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"}
Sep 14 08:20:41.726060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.726019    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp" event={"ID":"95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa","Type":"ContainerDied","Data":"bb7eae1d3a4c0ff8d6066df0dd3c6f59e5174e90ec9e2beda6b46da316ac68e6"}
Sep 14 08:20:41.726060 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.726049    2577 scope.go:117] "RemoveContainer" containerID="9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"
Sep 14 08:20:41.735846 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.735824    2577 scope.go:117] "RemoveContainer" containerID="9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"
Sep 14 08:20:41.736158 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:41.736137    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010\": container with ID starting with 9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010 not found: ID does not exist" containerID="9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"
Sep 14 08:20:41.736208 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.736165    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010"} err="failed to get container status \"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010\": rpc error: code = NotFound desc = could not find container \"9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010\": container with ID starting with 9d21b5cb28630bf020436626a373f29db776314bb7592c57e1df3a9a7bceb010 not found: ID does not exist"
Sep 14 08:20:41.748079 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.748050    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:20:41.752047 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.752027    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-7746e4-9bf484b48-k2djp"]
Sep 14 08:20:41.759969 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.759950    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" path="/var/lib/kubelet/pods/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa/volumes"
Sep 14 08:20:41.769592 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.769574    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:41.769592 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.769593    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:41.769709 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.769603    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpqdr\" (UniqueName: \"kubernetes.io/projected/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-kube-api-access-lpqdr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:41.769709 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:41.769613    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.306549 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.306514    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:20:59.307091 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.306731    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" podUID="120b1234-3f61-4655-b4f9-a42a4a22946c" containerName="maas-api" containerID="cri-o://a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625" gracePeriod=30
Sep 14 08:20:59.560246 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.560183    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:20:59.620780 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.620742    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") pod \"120b1234-3f61-4655-b4f9-a42a4a22946c\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") "
Sep 14 08:20:59.620983 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.620795    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rk589\" (UniqueName: \"kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589\") pod \"120b1234-3f61-4655-b4f9-a42a4a22946c\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") "
Sep 14 08:20:59.620983 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.620822    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs\") pod \"120b1234-3f61-4655-b4f9-a42a4a22946c\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") "
Sep 14 08:20:59.620983 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.620887    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca\") pod \"120b1234-3f61-4655-b4f9-a42a4a22946c\" (UID: \"120b1234-3f61-4655-b4f9-a42a4a22946c\") "
Sep 14 08:20:59.621302 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.621272    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "120b1234-3f61-4655-b4f9-a42a4a22946c" (UID: "120b1234-3f61-4655-b4f9-a42a4a22946c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:20:59.623038 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.623012    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589" (OuterVolumeSpecName: "kube-api-access-rk589") pod "120b1234-3f61-4655-b4f9-a42a4a22946c" (UID: "120b1234-3f61-4655-b4f9-a42a4a22946c"). InnerVolumeSpecName "kube-api-access-rk589". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:20:59.623134 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.623031    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "120b1234-3f61-4655-b4f9-a42a4a22946c" (UID: "120b1234-3f61-4655-b4f9-a42a4a22946c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:59.623134 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.623047    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "120b1234-3f61-4655-b4f9-a42a4a22946c" (UID: "120b1234-3f61-4655-b4f9-a42a4a22946c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:20:59.722618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.722581    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/120b1234-3f61-4655-b4f9-a42a4a22946c-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.722618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.722609    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.722618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.722619    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rk589\" (UniqueName: \"kubernetes.io/projected/120b1234-3f61-4655-b4f9-a42a4a22946c-kube-api-access-rk589\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.722618 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.722628    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/120b1234-3f61-4655-b4f9-a42a4a22946c-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:20:59.815037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.814963    2577 generic.go:356] "Generic (PLEG): container finished" podID="120b1234-3f61-4655-b4f9-a42a4a22946c" containerID="a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625" exitCode=0
Sep 14 08:20:59.815037 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.815018    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"
Sep 14 08:20:59.815210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.815042    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" event={"ID":"120b1234-3f61-4655-b4f9-a42a4a22946c","Type":"ContainerDied","Data":"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"}
Sep 14 08:20:59.815210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.815089    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv" event={"ID":"120b1234-3f61-4655-b4f9-a42a4a22946c","Type":"ContainerDied","Data":"6ccedeb5b544fbd65a6f20947ce6b999f2c4378059b84578c5cdd3cb7b2cf9de"}
Sep 14 08:20:59.815210 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.815110    2577 scope.go:117] "RemoveContainer" containerID="a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"
Sep 14 08:20:59.823838 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.823820    2577 scope.go:117] "RemoveContainer" containerID="a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"
Sep 14 08:20:59.824119 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:20:59.824101    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625\": container with ID starting with a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625 not found: ID does not exist" containerID="a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"
Sep 14 08:20:59.824198 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.824131    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625"} err="failed to get container status \"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625\": rpc error: code = NotFound desc = could not find container \"a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625\": container with ID starting with a18e3d1d4b9986d22142f442b959980a5583dcf292444b67b010364c7f97b625 not found: ID does not exist"
Sep 14 08:20:59.842396 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.842370    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:20:59.843997 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:20:59.843980    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-ea9d35-8996df44f-n74fv"]
Sep 14 08:21:01.760206 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:21:01.760175    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="120b1234-3f61-4655-b4f9-a42a4a22946c" path="/var/lib/kubelet/pods/120b1234-3f61-4655-b4f9-a42a4a22946c/volumes"
Sep 14 08:22:55.855351 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:22:55.855324    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:22:55.862201 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:22:55.862181    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-scw45_75740f09-a43d-494d-be3a-83ba623ef515/ovn-acl-logging/0.log"
Sep 14 08:25:29.292137 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292058    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292441    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8f4805-505f-4627-94d3-39a219961e19" containerName="payload-processing"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292454    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8f4805-505f-4627-94d3-39a219961e19" containerName="payload-processing"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292466    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292471    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292480    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" containerName="maas-api"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292487    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" containerName="maas-api"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292492    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292498    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292505    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292510    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292517    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292522    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292528    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292546    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="120b1234-3f61-4655-b4f9-a42a4a22946c" containerName="maas-api"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292551    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="120b1234-3f61-4655-b4f9-a42a4a22946c" containerName="maas-api"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292557    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d415092-bc78-4b37-812d-28322f71eb70" containerName="payload-pre-processing"
Sep 14 08:25:29.292603 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292562    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d415092-bc78-4b37-812d-28322f71eb70" containerName="payload-pre-processing"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292618    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292626    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292632    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c64a43f-aa33-4982-954d-ea4270310e83" containerName="istio-proxy"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292638    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="4d415092-bc78-4b37-812d-28322f71eb70" containerName="payload-pre-processing"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292643    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="120b1234-3f61-4655-b4f9-a42a4a22946c" containerName="maas-api"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292648    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="95d72e5c-d1e4-43a3-b8b2-e1e0242f1daa" containerName="maas-api"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292653    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292657    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292662    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b8f4805-505f-4627-94d3-39a219961e19" containerName="payload-processing"
Sep 14 08:25:29.293217 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.292667    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="70782bda-5bda-417a-8d1c-a9c06f005c7c" containerName="istio-proxy"
Sep 14 08:25:29.295669 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.295654    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.298299 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.298276    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-96ce3f1c-kserve-self-signed-certs\""
Sep 14 08:25:29.304973 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.304950    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:25:29.438281 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438253    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcn79\" (UniqueName: \"kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.438464 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438298    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.438464 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438333    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.438464 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438375    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.438464 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.438675 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.438526    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539417 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539386    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539428    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcn79\" (UniqueName: \"kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539458    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539483    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539501    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539612 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539526    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539942 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539898    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539942    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.539998 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.539976    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.541735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.541714    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.541903 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.541886    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.551725 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.551666    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcn79\" (UniqueName: \"kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79\") pod \"e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") " pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.605820 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.605795    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:29.875416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:29.875394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:25:29.877329 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:25:29.877299    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88fb0b2f_c9bb_4bf8_a594_5e05a234f088.slice/crio-3ee69f35bc394233b6823c5906ba99d98e21436b67ed3d3864c44e3883eb1823 WatchSource:0}: Error finding container 3ee69f35bc394233b6823c5906ba99d98e21436b67ed3d3864c44e3883eb1823: Status 404 returned error can't find the container with id 3ee69f35bc394233b6823c5906ba99d98e21436b67ed3d3864c44e3883eb1823
Sep 14 08:25:30.054850 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:30.054812    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" event={"ID":"88fb0b2f-c9bb-4bf8-a594-5e05a234f088","Type":"ContainerStarted","Data":"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"}
Sep 14 08:25:30.055042 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:30.054861    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" event={"ID":"88fb0b2f-c9bb-4bf8-a594-5e05a234f088","Type":"ContainerStarted","Data":"3ee69f35bc394233b6823c5906ba99d98e21436b67ed3d3864c44e3883eb1823"}
Sep 14 08:25:30.072596 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:30.072550    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" podStartSLOduration=1.072533825 podStartE2EDuration="1.072533825s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:25:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:25:30.071082615 +0000 UTC m=+2254.845580963" watchObservedRunningTime="2026-09-14 08:25:30.072533825 +0000 UTC m=+2254.847032160"
Sep 14 08:25:39.606745 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:39.606705    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:39.607357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:39.606771    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:39.619099 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:39.619075    2577 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:25:40.110679 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:25:40.110647    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:26:13.678311 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:13.678275    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:26:28.698369 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:28.698310    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" podUID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" containerName="main" containerID="cri-o://8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664" gracePeriod=45
Sep 14 08:26:28.943797 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:28.943774    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:26:29.071120 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071037    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071120 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071093    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071133    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcn79\" (UniqueName: \"kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071184    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071296    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071340 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071335    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home\") pod \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\" (UID: \"88fb0b2f-c9bb-4bf8-a594-5e05a234f088\") "
Sep 14 08:26:29.071577 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071400    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:29.071577 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071539    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache" (OuterVolumeSpecName: "model-cache") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:29.071677 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071621    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home" (OuterVolumeSpecName: "home") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:29.071729 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071717    2577 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tmp-dir\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.071768 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071731    2577 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-model-cache\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.071768 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.071740    2577 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-home\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.073395 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.073374    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79" (OuterVolumeSpecName: "kube-api-access-bcn79") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "kube-api-access-bcn79". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:26:29.073591 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.073568    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm" (OuterVolumeSpecName: "dshm") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 14 08:26:29.073685 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.073590    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "88fb0b2f-c9bb-4bf8-a594-5e05a234f088" (UID: "88fb0b2f-c9bb-4bf8-a594-5e05a234f088"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:29.172664 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.172640    2577 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-tls-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.172664 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.172662    2577 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-dshm\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.172826 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.172673    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcn79\" (UniqueName: \"kubernetes.io/projected/88fb0b2f-c9bb-4bf8-a594-5e05a234f088-kube-api-access-bcn79\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:29.337793 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.337698    2577 generic.go:356] "Generic (PLEG): container finished" podID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" containerID="8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664" exitCode=0
Sep 14 08:26:29.337793 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.337741    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"
Sep 14 08:26:29.338006 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.337783    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" event={"ID":"88fb0b2f-c9bb-4bf8-a594-5e05a234f088","Type":"ContainerDied","Data":"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"}
Sep 14 08:26:29.338006 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.337829    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2" event={"ID":"88fb0b2f-c9bb-4bf8-a594-5e05a234f088","Type":"ContainerDied","Data":"3ee69f35bc394233b6823c5906ba99d98e21436b67ed3d3864c44e3883eb1823"}
Sep 14 08:26:29.338006 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.337850    2577 scope.go:117] "RemoveContainer" containerID="8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"
Sep 14 08:26:29.347436 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.347415    2577 scope.go:117] "RemoveContainer" containerID="8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"
Sep 14 08:26:29.347707 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:26:29.347688    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664\": container with ID starting with 8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664 not found: ID does not exist" containerID="8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"
Sep 14 08:26:29.347775 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.347715    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664"} err="failed to get container status \"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664\": rpc error: code = NotFound desc = could not find container \"8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664\": container with ID starting with 8a83e43dbb83af52b980790e691dc755b18ed431ea5aa8fb56d0a57077a98664 not found: ID does not exist"
Sep 14 08:26:29.360489 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.360464    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:26:29.363958 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.363938    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-96ce3f1c-kserve-799c65695-c4lx2"]
Sep 14 08:26:29.761837 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:29.761795    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" path="/var/lib/kubelet/pods/88fb0b2f-c9bb-4bf8-a594-5e05a234f088/volumes"
Sep 14 08:26:31.737716 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.737688    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"]
Sep 14 08:26:31.738086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.738062    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" containerName="main"
Sep 14 08:26:31.738086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.738076    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" containerName="main"
Sep 14 08:26:31.738164 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.738094    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:26:31.738164 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.738100    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="63cbea1c-1a8c-4d9b-9c6e-882262d5e386" containerName="istio-proxy"
Sep 14 08:26:31.738164 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.738162    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="88fb0b2f-c9bb-4bf8-a594-5e05a234f088" containerName="main"
Sep 14 08:26:31.740944 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.740928    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.752416 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.752394    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"]
Sep 14 08:26:31.795599 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.795573    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-metrics-certs\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.795751 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.795623    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vxbw4\" (UniqueName: \"kubernetes.io/projected/afc43f83-320d-49c4-b03c-730f3d3ad1c1-kube-api-access-vxbw4\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.795751 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.795672    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-maas-api-tls\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.795751 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.795716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc43f83-320d-49c4-b03c-730f3d3ad1c1-openshift-service-ca\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.896925 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.896895    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vxbw4\" (UniqueName: \"kubernetes.io/projected/afc43f83-320d-49c4-b03c-730f3d3ad1c1-kube-api-access-vxbw4\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.897061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.896952    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-maas-api-tls\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.897061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.896993    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc43f83-320d-49c4-b03c-730f3d3ad1c1-openshift-service-ca\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.897061 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.897012    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-metrics-certs\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.897713 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.897691    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/afc43f83-320d-49c4-b03c-730f3d3ad1c1-openshift-service-ca\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.899558 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.899536    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-maas-api-tls\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.899639 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.899546    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/afc43f83-320d-49c4-b03c-730f3d3ad1c1-metrics-certs\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:31.905662 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:31.905636    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vxbw4\" (UniqueName: \"kubernetes.io/projected/afc43f83-320d-49c4-b03c-730f3d3ad1c1-kube-api-access-vxbw4\") pod \"maas-api-8456c7bcb-zpcq9\" (UID: \"afc43f83-320d-49c4-b03c-730f3d3ad1c1\") " pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:32.051817 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:32.051738    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:32.326471 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:32.326429    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"]
Sep 14 08:26:32.329328 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:26:32.329299    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafc43f83_320d_49c4_b03c_730f3d3ad1c1.slice/crio-84c76b668e48e209369fd19ff3b10a98e3815dd1905cd490ee4340103299c172 WatchSource:0}: Error finding container 84c76b668e48e209369fd19ff3b10a98e3815dd1905cd490ee4340103299c172: Status 404 returned error can't find the container with id 84c76b668e48e209369fd19ff3b10a98e3815dd1905cd490ee4340103299c172
Sep 14 08:26:32.330673 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:32.330653    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 14 08:26:32.353875 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:32.353849    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9" event={"ID":"afc43f83-320d-49c4-b03c-730f3d3ad1c1","Type":"ContainerStarted","Data":"84c76b668e48e209369fd19ff3b10a98e3815dd1905cd490ee4340103299c172"}
Sep 14 08:26:33.360269 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:33.360185    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9" event={"ID":"afc43f83-320d-49c4-b03c-730f3d3ad1c1","Type":"ContainerStarted","Data":"7d16c50bd6b1bfd4dad74d339e6f8e03f32571635bbd2df6852e132e82cbb9a0"}
Sep 14 08:26:33.360630 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:33.360272    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:33.381704 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:33.381658    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9" podStartSLOduration=1.5190172180000001 podStartE2EDuration="2.381644475s" totalImagesPullingTime="862.627257ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:26:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:26:33.380011436 +0000 UTC m=+2318.154509771" watchObservedRunningTime="2026-09-14 08:26:33.381644475 +0000 UTC m=+2318.156142810"
Sep 14 08:26:39.370194 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:39.370162    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8456c7bcb-zpcq9"
Sep 14 08:26:46.952148 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:46.952113    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 08:26:46.952704 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:46.952307    2577 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" podUID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" containerName="maas-api" containerID="cri-o://685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b" gracePeriod=30
Sep 14 08:26:48.192276 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.192252    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 08:26:48.239489 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.239460    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") pod \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") "
Sep 14 08:26:48.239643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.239508    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xqqr\" (UniqueName: \"kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr\") pod \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") "
Sep 14 08:26:48.239643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.239575    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca\") pod \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") "
Sep 14 08:26:48.239643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.239591    2577 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") pod \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\" (UID: \"cbc1bdc9-ed11-411b-9057-e08b7784e77d\") "
Sep 14 08:26:48.239948 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.239906    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cbc1bdc9-ed11-411b-9057-e08b7784e77d" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 14 08:26:48.241609 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.241581    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cbc1bdc9-ed11-411b-9057-e08b7784e77d" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:48.241734 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.241714    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cbc1bdc9-ed11-411b-9057-e08b7784e77d" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 14 08:26:48.241734 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.241724    2577 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr" (OuterVolumeSpecName: "kube-api-access-2xqqr") pod "cbc1bdc9-ed11-411b-9057-e08b7784e77d" (UID: "cbc1bdc9-ed11-411b-9057-e08b7784e77d"). InnerVolumeSpecName "kube-api-access-2xqqr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 14 08:26:48.340260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.340173    2577 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-maas-api-tls\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:48.340260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.340203    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xqqr\" (UniqueName: \"kubernetes.io/projected/cbc1bdc9-ed11-411b-9057-e08b7784e77d-kube-api-access-2xqqr\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:48.340260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.340217    2577 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cbc1bdc9-ed11-411b-9057-e08b7784e77d-openshift-service-ca\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:48.340260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.340231    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cbc1bdc9-ed11-411b-9057-e08b7784e77d-metrics-certs\") on node \"ip-10-0-128-30.ec2.internal\" DevicePath \"\""
Sep 14 08:26:48.432283 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.432246    2577 generic.go:356] "Generic (PLEG): container finished" podID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" containerID="685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b" exitCode=0
Sep 14 08:26:48.432438 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.432296    2577 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"
Sep 14 08:26:48.432438 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.432295    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" event={"ID":"cbc1bdc9-ed11-411b-9057-e08b7784e77d","Type":"ContainerDied","Data":"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"}
Sep 14 08:26:48.432438 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.432419    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv" event={"ID":"cbc1bdc9-ed11-411b-9057-e08b7784e77d","Type":"ContainerDied","Data":"eb589366f7c9ddcc45256e06f5543e6037b2708e07b7b34513c3936c6586d6ed"}
Sep 14 08:26:48.432574 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.432449    2577 scope.go:117] "RemoveContainer" containerID="685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"
Sep 14 08:26:48.441688 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.441671    2577 scope.go:117] "RemoveContainer" containerID="685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"
Sep 14 08:26:48.441965 ip-10-0-128-30 kubenswrapper[2577]: E0914 08:26:48.441944    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b\": container with ID starting with 685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b not found: ID does not exist" containerID="685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"
Sep 14 08:26:48.442027 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.441975    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b"} err="failed to get container status \"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b\": rpc error: code = NotFound desc = could not find container \"685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b\": container with ID starting with 685abe4cf3d84bae87028ed6783a4ba10ff50794d41d55e657b94585116f7d3b not found: ID does not exist"
Sep 14 08:26:48.459073 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.459053    2577 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 08:26:48.463160 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:48.463140    2577 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-765c9ffcd4-f79dv"]
Sep 14 08:26:49.765070 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:26:49.765036    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" path="/var/lib/kubelet/pods/cbc1bdc9-ed11-411b-9057-e08b7784e77d/volumes"
Sep 14 08:27:01.659881 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:01.659846    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dd588448-bjbvq_a53d937d-5ca1-49f3-8312-27fa572b8b83/authorino/0.log"
Sep 14 08:27:08.020822 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:08.020788    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n9q5h_fde2699e-2559-4f06-a35f-56e4eabb8d73/manager/0.log"
Sep 14 08:27:08.138780 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:08.133229    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-6w2gs_70de97a6-a0b1-4859-b7cf-ee36d84c41b5/copy-manifests/0.log"
Sep 14 08:27:08.147585 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:08.147561    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-6w2gs_70de97a6-a0b1-4859-b7cf-ee36d84c41b5/manager/0.log"
Sep 14 08:27:08.854725 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:08.854695    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8456c7bcb-zpcq9_afc43f83-320d-49c4-b03c-730f3d3ad1c1/maas-api/0.log"
Sep 14 08:27:09.973687 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:09.973658    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dd588448-bjbvq_a53d937d-5ca1-49f3-8312-27fa572b8b83/authorino/0.log"
Sep 14 08:27:10.322010 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:10.321937    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-x87d6_bd7bf4c3-6882-4a2e-911f-0b08b5829b03/kuadrant-console-plugin/0.log"
Sep 14 08:27:11.276981 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:11.276948    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-ls8br_57ec6ca2-c92d-4e64-a704-e94f36ef1d0f/kube-auth-proxy/0.log"
Sep 14 08:27:11.622702 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:11.622630    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-x4j4k_62befff0-bbee-442a-9563-53438eeae0e9/payload-pre-processing/0.log"
Sep 14 08:27:12.559098 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:12.559069    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nnt76_ae8eacff-8d1b-40fa-b795-464ceb49179f/main/0.log"
Sep 14 08:27:12.890043 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:12.889963    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-cn8ft_10143ce2-e149-4b51-a9fd-6eb6c7e0a2ea/main/0.log"
Sep 14 08:27:18.659066 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.659032    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ds52d/must-gather-ts76d"]
Sep 14 08:27:18.659539 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.659390    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" containerName="maas-api"
Sep 14 08:27:18.659539 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.659402    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" containerName="maas-api"
Sep 14 08:27:18.659539 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.659461    2577 memory_manager.go:368] "RemoveStaleState removing state" podUID="cbc1bdc9-ed11-411b-9057-e08b7784e77d" containerName="maas-api"
Sep 14 08:27:18.662532 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.662515    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.666285 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.666262    2577 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-ds52d\"/\"default-dockercfg-s6bnm\""
Sep 14 08:27:18.667019 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.667001    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"kube-root-ca.crt\""
Sep 14 08:27:18.667791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.667767    2577 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-ds52d\"/\"openshift-service-ca.crt\""
Sep 14 08:27:18.677353 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.677328    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/must-gather-ts76d"]
Sep 14 08:27:18.722022 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.721987    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fed0264f-3a42-4f18-94da-e1a92f718833-must-gather-output\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.722165 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.722042    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xvfqt\" (UniqueName: \"kubernetes.io/projected/fed0264f-3a42-4f18-94da-e1a92f718833-kube-api-access-xvfqt\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.822824 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.822783    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fed0264f-3a42-4f18-94da-e1a92f718833-must-gather-output\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.822990 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.822836    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xvfqt\" (UniqueName: \"kubernetes.io/projected/fed0264f-3a42-4f18-94da-e1a92f718833-kube-api-access-xvfqt\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.823182 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.823161    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/fed0264f-3a42-4f18-94da-e1a92f718833-must-gather-output\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.832699 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.832681    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xvfqt\" (UniqueName: \"kubernetes.io/projected/fed0264f-3a42-4f18-94da-e1a92f718833-kube-api-access-xvfqt\") pod \"must-gather-ts76d\" (UID: \"fed0264f-3a42-4f18-94da-e1a92f718833\") " pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:18.971445 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:18.971352    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/must-gather-ts76d"
Sep 14 08:27:19.246286 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:19.246216    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/must-gather-ts76d"]
Sep 14 08:27:19.249423 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:27:19.249379    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfed0264f_3a42_4f18_94da_e1a92f718833.slice/crio-954f54c2a89c7dfe5d39fc8436ec57ba9bc06d77ce9b98dd5c602ceb35594347 WatchSource:0}: Error finding container 954f54c2a89c7dfe5d39fc8436ec57ba9bc06d77ce9b98dd5c602ceb35594347: Status 404 returned error can't find the container with id 954f54c2a89c7dfe5d39fc8436ec57ba9bc06d77ce9b98dd5c602ceb35594347
Sep 14 08:27:19.575632 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:19.575542    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/must-gather-ts76d" event={"ID":"fed0264f-3a42-4f18-94da-e1a92f718833","Type":"ContainerStarted","Data":"954f54c2a89c7dfe5d39fc8436ec57ba9bc06d77ce9b98dd5c602ceb35594347"}
Sep 14 08:27:20.585971 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:20.585326    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/must-gather-ts76d" event={"ID":"fed0264f-3a42-4f18-94da-e1a92f718833","Type":"ContainerStarted","Data":"b54e6ff6dd9b4af0e14f733422cd0b5d1c0d4ae8103857bd69b0eef4d8e7ceb4"}
Sep 14 08:27:20.585971 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:20.585382    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/must-gather-ts76d" event={"ID":"fed0264f-3a42-4f18-94da-e1a92f718833","Type":"ContainerStarted","Data":"ffcc550114da2c67b32f55bb5f09a314d2c7cfab7896b101017e01a206c2cd52"}
Sep 14 08:27:20.622116 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:20.622066    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ds52d/must-gather-ts76d" podStartSLOduration=1.71903129 podStartE2EDuration="2.622050986s" totalImagesPullingTime="903.019696ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-14 08:27:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:27:20.619162054 +0000 UTC m=+2365.393660388" watchObservedRunningTime="2026-09-14 08:27:20.622050986 +0000 UTC m=+2365.396549321"
Sep 14 08:27:21.996277 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:21.996238    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-77jmw_f868c088-452d-4089-bf15-4c76d7ed82a4/global-pull-secret-syncer/0.log"
Sep 14 08:27:22.385237 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:22.385146    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vwtrh_fc7c6845-1183-4552-9363-7deddb27a825/konnectivity-agent/0.log"
Sep 14 08:27:22.436714 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:22.436682    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-30.ec2.internal_6f4b283df53c7982920aa7f76f6f0d43/haproxy/0.log"
Sep 14 08:27:27.140258 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:27.140223    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-76dd588448-bjbvq_a53d937d-5ca1-49f3-8312-27fa572b8b83/authorino/0.log"
Sep 14 08:27:27.280418 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:27.280212    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-x87d6_bd7bf4c3-6882-4a2e-911f-0b08b5829b03/kuadrant-console-plugin/0.log"
Sep 14 08:27:28.807188 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.807139    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/alertmanager/0.log"
Sep 14 08:27:28.876333 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.876299    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/config-reloader/0.log"
Sep 14 08:27:28.904979 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.904950    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/kube-rbac-proxy-web/0.log"
Sep 14 08:27:28.928029 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.928005    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/kube-rbac-proxy/0.log"
Sep 14 08:27:28.949566 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.949538    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/kube-rbac-proxy-metric/0.log"
Sep 14 08:27:28.971432 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.971409    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/prom-label-proxy/0.log"
Sep 14 08:27:28.990535 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:28.990480    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_161ffc42-ea8b-4a77-abd4-eea3564164f3/init-config-reloader/0.log"
Sep 14 08:27:29.371404 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.371374    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvdm8_e4c95dba-db92-4922-8078-b7e25d11f8f0/node-exporter/0.log"
Sep 14 08:27:29.397791 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.397712    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvdm8_e4c95dba-db92-4922-8078-b7e25d11f8f0/kube-rbac-proxy/0.log"
Sep 14 08:27:29.420309 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.420277    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jvdm8_e4c95dba-db92-4922-8078-b7e25d11f8f0/init-textfile/0.log"
Sep 14 08:27:29.686368 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.686335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vpf27_64445ef6-a979-45ac-a513-a06ec6b5dd31/prometheus-operator/0.log"
Sep 14 08:27:29.707003 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.706967    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-68bcd7f987-vpf27_64445ef6-a979-45ac-a513-a06ec6b5dd31/kube-rbac-proxy/0.log"
Sep 14 08:27:29.733589 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.733557    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-pfpvq_5fbe172d-11a4-4e45-9703-940c9f3bab09/prometheus-operator-admission-webhook/0.log"
Sep 14 08:27:29.770599 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.770518    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d59cbb684-tz99m_d5c2c9f5-834e-481c-83b5-5791f22bff99/telemeter-client/0.log"
Sep 14 08:27:29.791968 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.791944    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d59cbb684-tz99m_d5c2c9f5-834e-481c-83b5-5791f22bff99/reload/0.log"
Sep 14 08:27:29.815637 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:29.815602    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6d59cbb684-tz99m_d5c2c9f5-834e-481c-83b5-5791f22bff99/kube-rbac-proxy/0.log"
Sep 14 08:27:31.278827 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:31.278794    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-5dcbc7df6b-8wn7n_eac7b0b4-5770-4b9d-9b66-552733b153ab/networking-console-plugin/0.log"
Sep 14 08:27:31.994341 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:31.994305    2577 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"]
Sep 14 08:27:32.002337 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.002310    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.007158 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.007013    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"]
Sep 14 08:27:32.071098 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.071066    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbzgt\" (UniqueName: \"kubernetes.io/projected/cd308814-a896-456d-af37-17d65154729b-kube-api-access-cbzgt\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.071457 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.071434    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-proc\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.071591 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.071574    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-lib-modules\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.071709 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.071696    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-sys\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.071822 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.071809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-podres\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.172853 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172818    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-proc\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.172853 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172857    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-lib-modules\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172881    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-sys\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172911    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-podres\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172984    2577 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbzgt\" (UniqueName: \"kubernetes.io/projected/cd308814-a896-456d-af37-17d65154729b-kube-api-access-cbzgt\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.172986    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-proc\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.173017    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-lib-modules\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173163 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.173119    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-podres\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.173534 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.173163    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cd308814-a896-456d-af37-17d65154729b-sys\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.181877 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.181842    2577 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbzgt\" (UniqueName: \"kubernetes.io/projected/cd308814-a896-456d-af37-17d65154729b-kube-api-access-cbzgt\") pod \"perf-node-gather-daemonset-8p4dq\" (UID: \"cd308814-a896-456d-af37-17d65154729b\") " pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.322789 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.322700    2577 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:32.511735 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.511708    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-76f99584f4-j9s58_fae60f99-2195-40c4-9d94-09a8bed79983/download-server/0.log"
Sep 14 08:27:32.620166 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.620138    2577 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"]
Sep 14 08:27:32.623700 ip-10-0-128-30 kubenswrapper[2577]: W0914 08:27:32.623669    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcd308814_a896_456d_af37_17d65154729b.slice/crio-727d8be7d5f5521336c259d3963741e71ca6acf1289086f73f60c907a94aac47 WatchSource:0}: Error finding container 727d8be7d5f5521336c259d3963741e71ca6acf1289086f73f60c907a94aac47: Status 404 returned error can't find the container with id 727d8be7d5f5521336c259d3963741e71ca6acf1289086f73f60c907a94aac47
Sep 14 08:27:32.687728 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:32.687694    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq" event={"ID":"cd308814-a896-456d-af37-17d65154729b","Type":"ContainerStarted","Data":"727d8be7d5f5521336c259d3963741e71ca6acf1289086f73f60c907a94aac47"}
Sep 14 08:27:33.699260 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:33.699213    2577 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq" event={"ID":"cd308814-a896-456d-af37-17d65154729b","Type":"ContainerStarted","Data":"e3f1cb6325e8e2dcba850927c963049b093f8058ada268b4c1ae4b1c82950fe5"}
Sep 14 08:27:33.700092 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:33.700067    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:33.719907 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:33.719845    2577 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq" podStartSLOduration=2.719825108 podStartE2EDuration="2.719825108s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-14 08:27:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-14 08:27:33.718594212 +0000 UTC m=+2378.493092547" watchObservedRunningTime="2026-09-14 08:27:33.719825108 +0000 UTC m=+2378.494323443"
Sep 14 08:27:34.377136 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:34.377110    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6jdf7_cae33c98-1e21-45e3-8f8e-fce2594b30f0/dns/0.log"
Sep 14 08:27:34.407355 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:34.407323    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-6jdf7_cae33c98-1e21-45e3-8f8e-fce2594b30f0/kube-rbac-proxy/0.log"
Sep 14 08:27:34.601374 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:34.601346    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cwmch_9ac03f35-5505-4be8-b807-f27b60d3b440/dns-node-resolver/0.log"
Sep 14 08:27:35.229275 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:35.229231    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jn5w7_ad93e5d6-b44c-4e63-9745-a431667d9410/node-ca/0.log"
Sep 14 08:27:36.722638 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:36.722608    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-ls8br_57ec6ca2-c92d-4e64-a704-e94f36ef1d0f/kube-auth-proxy/0.log"
Sep 14 08:27:36.872904 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:36.872865    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-x4j4k_62befff0-bbee-442a-9563-53438eeae0e9/payload-pre-processing/0.log"
Sep 14 08:27:37.838287 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:37.838257    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-mrldc_97706785-9dd7-4198-90cd-9addb0678bc0/serve-healthcheck-canary/0.log"
Sep 14 08:27:38.306182 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:38.306147    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-zp4c6_23231f53-30d5-40c0-b1d8-41c236fcb9c6/insights-operator/0.log"
Sep 14 08:27:38.306643 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:38.306625    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5844b856db-zp4c6_23231f53-30d5-40c0-b1d8-41c236fcb9c6/insights-operator/1.log"
Sep 14 08:27:38.481295 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:38.481267    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5rtk_c4d066a0-bda0-4c44-95c4-823e1a46bd83/kube-rbac-proxy/0.log"
Sep 14 08:27:38.501783 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:38.501750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5rtk_c4d066a0-bda0-4c44-95c4-823e1a46bd83/exporter/0.log"
Sep 14 08:27:38.524086 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:38.524064    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-x5rtk_c4d066a0-bda0-4c44-95c4-823e1a46bd83/extractor/0.log"
Sep 14 08:27:40.725186 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:40.725150    2577 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-ds52d/perf-node-gather-daemonset-8p4dq"
Sep 14 08:27:41.035248 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:41.035167    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-n9q5h_fde2699e-2559-4f06-a35f-56e4eabb8d73/manager/0.log"
Sep 14 08:27:41.127630 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:41.127595    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-6w2gs_70de97a6-a0b1-4859-b7cf-ee36d84c41b5/manager/0.log"
Sep 14 08:27:41.147974 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:41.147911    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-84c4c5b687-6w2gs_70de97a6-a0b1-4859-b7cf-ee36d84c41b5/copy-manifests/0.log"
Sep 14 08:27:42.440409 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:42.440384    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-ls8br_57ec6ca2-c92d-4e64-a704-e94f36ef1d0f/kube-auth-proxy/0.log"
Sep 14 08:27:42.517598 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:42.517562    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-x4j4k_62befff0-bbee-442a-9563-53438eeae0e9/payload-pre-processing/0.log"
Sep 14 08:27:42.678357 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:42.678328    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-784955c84b-bblsp_f91d4d29-0894-4215-b7c4-6cbd17b73bc5/manager/0.log"
Sep 14 08:27:44.541293 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:44.541209    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-86f4846dcd-ls8br_57ec6ca2-c92d-4e64-a704-e94f36ef1d0f/kube-auth-proxy/0.log"
Sep 14 08:27:44.611870 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:44.611844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-x4j4k_62befff0-bbee-442a-9563-53438eeae0e9/payload-pre-processing/0.log"
Sep 14 08:27:49.306874 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.306840    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-5gv2g_0d86d587-f44a-4edb-9a39-0f33f53d3909/kube-multus/0.log"
Sep 14 08:27:49.373616 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.373587    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/kube-multus-additional-cni-plugins/0.log"
Sep 14 08:27:49.399069 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.399039    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/egress-router-binary-copy/0.log"
Sep 14 08:27:49.423034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.423004    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/cni-plugins/0.log"
Sep 14 08:27:49.446584 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.446554    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/bond-cni-plugin/0.log"
Sep 14 08:27:49.467034 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.467014    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/routeoverride-cni/0.log"
Sep 14 08:27:49.491522 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.491501    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/whereabouts-cni-bincopy/0.log"
Sep 14 08:27:49.511506 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.511484    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-nwd45_ed74e3c7-89d0-434c-800a-df7e4c2d75d7/whereabouts-cni/0.log"
Sep 14 08:27:49.978790 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.978759    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-knfrc_468850b2-f30d-49d0-aa49-7be726bc68d7/network-metrics-daemon/0.log"
Sep 14 08:27:49.993492 ip-10-0-128-30 kubenswrapper[2577]: I0914 08:27:49.993469    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-knfrc_468850b2-f30d-49d0-aa49-7be726bc68d7/kube-rbac-proxy/0.log"