Sep 09 12:46:23.161801 ip-10-0-137-1 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 12:46:23.161816 ip-10-0-137-1 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 12:46:23.161826 ip-10-0-137-1 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 12:46:23.162057 ip-10-0-137-1 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 12:46:34.807691 ip-10-0-137-1 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 12:46:34.807709 ip-10-0-137-1 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot ab30259372884fdca42fd678a0ce1e8d --
Sep 09 12:48:21.850752 ip-10-0-137-1 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 12:48:22.280214 ip-10-0-137-1 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:22.280214 ip-10-0-137-1 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 12:48:22.280214 ip-10-0-137-1 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:22.280214 ip-10-0-137-1 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 12:48:22.280214 ip-10-0-137-1 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 12:48:22.280896 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.280275    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 12:48:22.284446 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284430    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.284446 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284446    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284450    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284453    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284456    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284460    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284462    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284465    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284468    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284470    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284473    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284476    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284478    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284481    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284483    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284486    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284489    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284491    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284494    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284496    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284510    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.284515 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284516    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284519    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284522    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284525    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284528    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284531    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284534    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284536    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284539    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284541    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284544    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284546    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284549    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284551    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284554    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284556    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284559    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284561    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284564    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284566    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.285000 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284569    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284571    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284574    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284576    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284579    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284581    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284584    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284586    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284589    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284591    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284594    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284598    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284601    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284604    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284609    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284614    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284617    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284620    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284622    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284625    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.285482 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284628    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284630    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284633    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284637    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284641    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284644    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284647    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284650    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284653    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284655    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284658    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284661    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284664    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284667    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284670    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284673    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284675    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284678    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284680    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.286021 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284683    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284685    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284688    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284690    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284693    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284697    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284700    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284704    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284707    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284709    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284712    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284715    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284717    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284720    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284722    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284725    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284727    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284730    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.284734    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285420    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.286688 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285426    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285430    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285433    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285437    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285440    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285443    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285446    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285449    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285452    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285455    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285458    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285461    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285464    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285467    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285469    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285472    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285474    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285478    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285480    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285484    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.287364 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285488    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285491    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285494    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285496    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285517    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285520    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285522    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285525    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285528    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285530    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285533    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285536    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285538    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285541    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285543    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285546    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285551    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285554    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285557    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.287881 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285560    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285562    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285565    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285568    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285570    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285573    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285575    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285578    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285581    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285583    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285586    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285589    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285591    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285594    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285597    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285600    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285603    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285606    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285609    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285612    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.288341 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285615    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285617    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285621    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285623    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285626    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285628    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285632    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285635    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285638    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285641    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285644    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285647    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285650    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285653    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285656    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285658    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285661    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285664    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285677    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285680    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.288855 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285682    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285685    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285688    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285691    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285694    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285697    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285699    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285702    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285705    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285708    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285710    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285715    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285718    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285721    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285724    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285728    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285731    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285734    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.285737    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.289342 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286516    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286527    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286532    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286536    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286541    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286546    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286551    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286556    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286560    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286565    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286570    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286579    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286584    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286588    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286592    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286597    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286601    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286605    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286610    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286614    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.289825 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286619    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286624    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286628    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286632    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286646    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286651    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286655    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286659    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286664    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286668    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286675    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286681    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286686    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286691    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286696    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286700    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286704    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286714    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286718    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.290322 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286723    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286727    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286732    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286736    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286742    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286746    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286750    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286754    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286760    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286764    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286773    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286777    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286781    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286786    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286791    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286795    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286799    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286804    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286808    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286812    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.290807 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286816    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286820    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286825    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286834    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286838    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286841    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286846    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286850    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286854    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286858    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286862    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286866    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286871    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286875    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286879    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286887    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286891    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286896    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286899    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286903    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.291315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286909    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286913    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286920    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286924    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286928    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286932    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286937    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286946    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286951    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286955    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286960    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286964    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286969    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286973    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.286977    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287006    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287118    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287124    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287128    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.291811 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287131    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287190    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287224    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287231    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287235    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287240    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287243    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287247    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287251    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287254    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287257    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287260    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287264    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287269    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287272    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287274    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287277    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287280    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287283    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287308    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287312    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287315    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287319    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287323    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 12:48:22.292271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287327    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287330    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287333    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287336    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287339    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287342    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287345    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287348    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287351    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287355    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287358    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287361    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287363    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287366    2580 flags.go:64] FLAG: --enable-server="true"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287369    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287374    2580 flags.go:64] FLAG: --event-burst="100"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287377    2580 flags.go:64] FLAG: --event-qps="50"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287380    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287384    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287387    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287395    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287400    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287403    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287406    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287409    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 12:48:22.292888 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287412    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287414    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287417    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287420    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287423    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287425    2580 flags.go:64] FLAG: --feature-gates=""
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287429    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287432    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287435    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287438    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287441    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287444    2580 flags.go:64] FLAG: --help="false"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287447    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287450    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287453    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287456    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287459    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287463    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287465    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287468    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287471    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287474    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287477    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287480    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 12:48:22.293493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287483    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287485    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287488    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287491    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287495    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287498    2580 flags.go:64] FLAG: --lock-file=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287516    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287519    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287521    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287527    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287530    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287534    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287536    2580 flags.go:64] FLAG: --logging-format="text"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287541    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287544    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287547    2580 flags.go:64] FLAG: --manifest-url=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287549    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287554    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287556    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287560    2580 flags.go:64] FLAG: --max-pods="110"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287563    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287566    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287569    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287572    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287575    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 12:48:22.294143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287578    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287580    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287588    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287591    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287594    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287597    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287600    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287605    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287608    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287612    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287614    2580 flags.go:64] FLAG: --port="10250"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287617    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287622    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ea3a30de3577093c"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287625    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287628    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287631    2580 flags.go:64] FLAG: --register-node="true"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287633    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287637    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287640    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287643    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287647    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287650    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287653    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287656    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287659    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 12:48:22.294821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287662    2580 flags.go:64] FLAG: --runonce="false"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287664    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287667    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287670    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287673    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287676    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287679    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287682    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287685    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287688    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287690    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287693    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287697    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287700    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287703    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287705    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287711    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287714    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287717    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287723    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287726    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287729    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287731    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287734    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287737    2580 flags.go:64] FLAG: --v="2"
Sep 09 12:48:22.295427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287741    2580 flags.go:64] FLAG: --version="false"
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287745    2580 flags.go:64] FLAG: --vmodule=""
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287750    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.287753    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287864    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287868    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287871    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287874    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287877    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287880    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287883    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287885    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287888    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287891    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287894    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287896    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287899    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287902    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287904    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287907    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.296058 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287909    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287912    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287915    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287917    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287920    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287922    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287926    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287929    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287931    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287934    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287936    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287939    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287941    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287944    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287948    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287952    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287954    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287957    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287961    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.296548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287965    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287968    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287970    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287973    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287976    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287978    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287981    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287984    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287986    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287989    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287991    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287994    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287996    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.287999    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288001    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288004    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288006    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288009    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288012    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288016    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.297027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288018    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288021    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288023    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288026    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288028    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288031    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288033    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288037    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288040    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288043    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288045    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288048    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288050    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288053    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288055    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288059    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288062    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288065    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288068    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.297556 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288070    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288073    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288075    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288078    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288080    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288083    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288085    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288088    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288090    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288093    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288095    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288098    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288102    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288105    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288108    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288110    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288113    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288115    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288119    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288122    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.298018 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288127    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288130    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288133    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288136    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.288139    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.288617    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.295230    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.295244    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.296800    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.296808    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296870    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296875    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296879    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296882    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296886    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.298546 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296889    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296892    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296894    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296897    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296900    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296903    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296906    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296908    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296911    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296914    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296916    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296919    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296921    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296924    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296927    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296929    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296931    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296934    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296937    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296940    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.298921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296943    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296946    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296948    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296951    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296955    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296958    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296962    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296966    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296969    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296973    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296975    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296978    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296981    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296984    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296987    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296990    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296992    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296995    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.296997    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.299399 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297000    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297003    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297005    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297008    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297011    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297013    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297016    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297018    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297021    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297023    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297026    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297028    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297031    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297034    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297037    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297040    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297043    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297046    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297048    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297051    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.299867 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297053    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297056    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297058    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297061    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297064    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297067    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297069    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297072    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297074    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297077    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297079    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297082    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297085    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297089    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297093    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297096    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297099    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297102    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297104    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297107    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.300354 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297110    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297112    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297115    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297117    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297120    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297123    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297126    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297129    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297132    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297134    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297137    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297139    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297142    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297144    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297148    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.300853 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.297153    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297257    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297261    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297264    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297267    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297270    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297274    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297278    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297281    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297284    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297286    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297289    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297292    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297294    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297297    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297299    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297302    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297305    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297307    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 12:48:22.301219 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297310    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297312    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297315    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297317    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297320    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297323    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297326    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297328    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297331    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297333    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297336    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297339    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297341    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297343    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297346    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297349    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297351    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297354    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297356    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297359    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 12:48:22.301701 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297361    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297364    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297366    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297368    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297371    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297373    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297376    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297378    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297381    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297383    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297386    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297388    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297390    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297393    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297395    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297398    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297401    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297403    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297406    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297409    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 12:48:22.302186 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297412    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297414    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297417    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297419    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297422    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297424    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297431    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297433    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297436    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297438    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297440    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297444    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297447    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297450    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297453    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297455    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297458    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297461    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297463    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 12:48:22.302674 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297465    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297468    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297470    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297473    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297476    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297479    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297481    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297484    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297486    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297490    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297493    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297496    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297513    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297517    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297519    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297522    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297525    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297528    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297530    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 12:48:22.303129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297534    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297537    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:22.297539    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.297544    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.297635    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.300063    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.300941    2580 server.go:1019] "Starting client certificate rotation"
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.301043    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:48:22.303733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.301945    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 12:48:22.304198 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.304184    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:48:22.305379 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.305362    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 12:48:22.315487 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.315470    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 09 12:48:22.318121 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.318107    2580 log.go:25] "Validated CRI v1 image API"
Sep 09 12:48:22.319017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.318992    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 12:48:22.324961 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.324930    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 83cd703d-c504-4b97-80cb-29493e5460e2:/dev/nvme0n1p3 e733b16e-df92-42ab-9ade-833c83feb404:/dev/nvme0n1p4]
Sep 09 12:48:22.325025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.324960    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 12:48:22.331369 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.331350    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:48:22.331627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.331520    2580 manager.go:216] Machine: {Timestamp:2026-09-09 12:48:22.330235732 +0000 UTC m=+0.365483629 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102122 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e7b3492164bb2e35bdff810b9be4c SystemUUID:ec2e7b34-9216-4bb2-e35b-dff810b9be4c BootID:ab302593-7288-4fdc-a42f-d678a0ce1e8d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:55:79:b2:bd:a7 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:55:79:b2:bd:a7 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:59:09:34:29:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 12:48:22.331627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.331626    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 12:48:22.331724 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.331701    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 12:48:22.335268 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.335246    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 12:48:22.335394 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.335270    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-1.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 12:48:22.336022 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.336012    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 12:48:22.336053 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.336025    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 12:48:22.336053 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.336032    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:48:22.336053 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.336045    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 12:48:22.336775 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.336766    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 12:48:22.339218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339207    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:48:22.339318 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339309    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 12:48:22.339362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339336    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 12:48:22.339362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339345    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 12:48:22.339362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339355    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 12:48:22.339462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339364    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 09 12:48:22.339462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339378    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 12:48:22.339736 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339724    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:48:22.339769 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.339749    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 12:48:22.342095 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.342079    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 12:48:22.344219 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.344205    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 12:48:22.344282 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.344234    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 12:48:22.345274 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345260    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 12:48:22.345316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345281    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 12:48:22.345316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345291    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 12:48:22.345316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345299    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345323    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345333    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345353    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345361    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345370    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345378    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 12:48:22.345403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345397    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 12:48:22.345651 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.345405    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 12:48:22.346987 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.346975    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 12:48:22.346987 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.346988    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 12:48:22.350700 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.350681    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-137-1.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 12:48:22.350795 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.350784    2580 server.go:1268] "Started kubelet"
Sep 09 12:48:22.350944 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.350863    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 12:48:22.351091 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.351053    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 12:48:22.351156 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.351102    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 12:48:22.351278 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.351242    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 12:48:22.351368 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.351250    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-137-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 12:48:22.352032 ip-10-0-137-1 systemd[1]: Started Kubernetes Kubelet.
Sep 09 12:48:22.352651 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.352637    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 12:48:22.354377 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.354363    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 12:48:22.356451 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.356433    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 12:48:22.357424 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.357406    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 12:48:22.357521 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.357427    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 12:48:22.357662 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.357644    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.358213 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.358195    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 12:48:22.358298 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.358286    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 12:48:22.358371 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.358345    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 12:48:22.358428 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.358410    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 12:48:22.358428 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.358419    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 12:48:22.360482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360459    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 12:48:22.360482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360479    2580 factory.go:55] Registering systemd factory
Sep 09 12:48:22.360482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360489    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 09 12:48:22.360702 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.360459    2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-137-1.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 09 12:48:22.360875 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.360825    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 09 12:48:22.360969 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360952    2580 factory.go:153] Registering CRI-O factory
Sep 09 12:48:22.361017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360975    2580 factory.go:223] Registration of the crio container factory successfully
Sep 09 12:48:22.361017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.360994    2580 factory.go:103] Registering Raw factory
Sep 09 12:48:22.361017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.361009    2580 manager.go:1196] Started watching for new ooms in manager
Sep 09 12:48:22.361426 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.361413    2580 manager.go:349] Starting recovery of all containers
Sep 09 12:48:22.362003 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.360582    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-1.ec2.internal.18d3a7ea70ac801d  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-1.ec2.internal,UID:ip-10-0-137-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-137-1.ec2.internal,},FirstTimestamp:2026-09-09 12:48:22.350757917 +0000 UTC m=+0.386005978,LastTimestamp:2026-09-09 12:48:22.350757917 +0000 UTC m=+0.386005978,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-1.ec2.internal,}"
Sep 09 12:48:22.373817 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.373688    2580 manager.go:354] Recovery completed
Sep 09 12:48:22.377927 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.377916    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.379707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.379686    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.379797 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.379717    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.379797 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.379729    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.380342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.380325    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 12:48:22.380342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.380341    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 12:48:22.380472 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.380346    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.380472 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.380359    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 12:48:22.381082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381067    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.381188 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381091    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.381188 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381100    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.381621 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381607    2580 policy_none.go:49] "None policy: Start"
Sep 09 12:48:22.381621 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381621    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 12:48:22.381730 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.381630    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 12:48:22.381962 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.381890    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-1.ec2.internal.18d3a7ea726632ed  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-1.ec2.internal,UID:ip-10-0-137-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-137-1.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-137-1.ec2.internal,},FirstTimestamp:2026-09-09 12:48:22.379705069 +0000 UTC m=+0.414952966,LastTimestamp:2026-09-09 12:48:22.379705069 +0000 UTC m=+0.414952966,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-1.ec2.internal,}"
Sep 09 12:48:22.382555 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.382545    2580 policy_none.go:47] "Start"
Sep 09 12:48:22.390323 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.390303    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x7bvw"
Sep 09 12:48:22.392853 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.392787    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-137-1.ec2.internal.18d3a7ea726675a1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-137-1.ec2.internal,UID:ip-10-0-137-1.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-137-1.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-137-1.ec2.internal,},FirstTimestamp:2026-09-09 12:48:22.379722145 +0000 UTC m=+0.414970042,LastTimestamp:2026-09-09 12:48:22.379722145 +0000 UTC m=+0.414970042,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-137-1.ec2.internal,}"
Sep 09 12:48:22.396381 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.396361    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-x7bvw"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414176    2580 manager.go:337] "Starting Device Plugin manager"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.414211    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414220    2580 server.go:87] "Starting device plugin registration server"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414469    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414482    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414572    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414641    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.414649    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.415201    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 12:48:22.423570 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.415249    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.444587 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.444553    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 12:48:22.445818 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.445802    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 12:48:22.445884 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.445843    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 12:48:22.445884 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.445862    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 12:48:22.445965 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.445896    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 12:48:22.448260 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.448243    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:22.515571 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.515552    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.516879 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.516863    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.516940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.516910    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.516940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.516925    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.517030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.516950    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.527127 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.527113    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.527193 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.527131    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-137-1.ec2.internal\": node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.546288 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.546211    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"]
Sep 09 12:48:22.546288 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.546284    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.547593 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.547576    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.547659 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.547609    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.547659 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.547624    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.549822 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.549810    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.549951 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.549934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.549989 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.549961    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.550482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550462    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.550590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550466    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.550590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550518    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.550590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550531    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.550590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550536    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.550590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.550554    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.551381 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.551364    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.552656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.552639    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.552736 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.552667    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:22.553372 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.553358    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:22.553431 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.553385    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:22.553431 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.553398    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:22.559530 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.559514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.559618 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.559566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.559618 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.559614    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/270358c4069fad120920bbf1f2675138-config\") pod \"kube-apiserver-proxy-ip-10-0-137-1.ec2.internal\" (UID: \"270358c4069fad120920bbf1f2675138\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.579066 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.579052    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-1.ec2.internal\" not found" node="ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.583297 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.583281    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-137-1.ec2.internal\" not found" node="ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.651830 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.651810    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.660650 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660632    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.660703 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.660703 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/270358c4069fad120920bbf1f2675138-config\") pod \"kube-apiserver-proxy-ip-10-0-137-1.ec2.internal\" (UID: \"270358c4069fad120920bbf1f2675138\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.660802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/270358c4069fad120920bbf1f2675138-config\") pod \"kube-apiserver-proxy-ip-10-0-137-1.ec2.internal\" (UID: \"270358c4069fad120920bbf1f2675138\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.660802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.660802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.660757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d936d2fc428f3884c06a358b12bfbd88-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal\" (UID: \"d936d2fc428f3884c06a358b12bfbd88\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.752558 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.752537    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.853224 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.853165    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:22.881692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.881673    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.886229 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:22.886215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:22.953628 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:22.953597    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.054067 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.054042    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.154911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.154854    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.174209 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.174184    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:23.254958 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.254930    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.303070 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.303040    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 12:48:23.303464 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.303170    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:48:23.303464 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.303209    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 12:48:23.355213 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.355190    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.402943 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.402910    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 12:43:23 +0000 UTC" deadline="2028-02-07 00:17:20.299978303 +0000 UTC"
Sep 09 12:48:23.402943 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.402938    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12371h28m56.897043339s"
Sep 09 12:48:23.426637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.426585    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 12:48:23.427588 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.427562    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 12:48:23.427656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.427592    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 12:48:23.427656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.427607    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeHasSufficientPID"
Sep 09 12:48:23.455404 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.455382    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.479840 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.479823    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:23.557853 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.557824    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.659265 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.659240    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.759897 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.759879    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.860330 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:23.860303    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-137-1.ec2.internal\" not found"
Sep 09 12:48:23.884154 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:23.884128    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod270358c4069fad120920bbf1f2675138.slice/crio-2837c6a95b74f0fadc42163e21e62ddbf823141167a5ece48f01784146642377 WatchSource:0}: Error finding container 2837c6a95b74f0fadc42163e21e62ddbf823141167a5ece48f01784146642377: Status 404 returned error can't find the container with id 2837c6a95b74f0fadc42163e21e62ddbf823141167a5ece48f01784146642377
Sep 09 12:48:23.891480 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.891462    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:48:23.892741 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:23.892719    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd936d2fc428f3884c06a358b12bfbd88.slice/crio-4fd9b05c0005ed47c8c44ea1f1a19d6b05ecf6961525052c7e43c4397fd4c6c9 WatchSource:0}: Error finding container 4fd9b05c0005ed47c8c44ea1f1a19d6b05ecf6961525052c7e43c4397fd4c6c9: Status 404 returned error can't find the container with id 4fd9b05c0005ed47c8c44ea1f1a19d6b05ecf6961525052c7e43c4397fd4c6c9
Sep 09 12:48:23.941577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.941552    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:23.958092 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.958067    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:23.973059 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.973039    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:48:23.974131 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.974116    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal"
Sep 09 12:48:23.984948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:23.984930    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 12:48:24.277735 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.277714    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:24.340334 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.340305    2580 apiserver.go:52] "Watching apiserver"
Sep 09 12:48:24.348351 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.348334    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 12:48:24.348723 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.348701    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-rc72g","openshift-multus/multus-additional-cni-plugins-rjk6k","openshift-multus/multus-v5mbc","openshift-multus/network-metrics-daemon-q7pfr","kube-system/konnectivity-agent-2f5np","openshift-cluster-node-tuning-operator/tuned-p6f5w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal","openshift-network-diagnostics/network-check-target-hkb9w","openshift-network-operator/iptables-alerter-whwqr","openshift-ovn-kubernetes/ovnkube-node-qdwjh","kube-system/global-pull-secret-syncer-fj9l6","kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7","openshift-dns/node-resolver-8ngjq"]
Sep 09 12:48:24.351673 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.351654    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.353933 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.353906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.354578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354450    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.354578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354465    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.354578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354510    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 12:48:24.354578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354519    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 12:48:24.354578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354465    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 12:48:24.354892 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.354795    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 12:48:24.355286 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.355267    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-fm9qw\""
Sep 09 12:48:24.356638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.356333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.356638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.356348    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 12:48:24.356638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.356380    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.356638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.356377    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.356638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.356465    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-t65pf\""
Sep 09 12:48:24.358691 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.358671    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-qjprr\""
Sep 09 12:48:24.358800 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.358783    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 12:48:24.358898 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.358883    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.359151 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.359133    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 12:48:24.359230 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.359154    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 12:48:24.359610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.359567    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.360090 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.360066    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.360187 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.360160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:24.362766 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.362746    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.364944 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.364930    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 12:48:24.365191 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.365173    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 12:48:24.365242 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.365194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-ckzvl\""
Sep 09 12:48:24.365346 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.365331    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.367825 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.367807    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.367948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.367927    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-5v55n\""
Sep 09 12:48:24.368054 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.368029    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.368262 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.368239    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.369026 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369005    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-script-lib\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cnibin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-socket-dir-parent\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369098    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-netd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-env-overrides\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-netns\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369167    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-multus\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369222    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-kubelet\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369244    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-netns\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369271    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-systemd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-var-lib-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-etc-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369377    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-ovn\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369394    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369410    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-bin\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369424    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgllp\" (UniqueName: \"kubernetes.io/projected/333cd3fd-e195-43da-95c1-df8ba568219f-kube-api-access-tgllp\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cni-binary-copy\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369455    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-kubelet\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-conf-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-slash\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369559 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-log-socket\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369577    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369623    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-config\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/333cd3fd-e195-43da-95c1-df8ba568219f-ovn-node-metrics-cert\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369661    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-k8s-cni-cncf-io\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-bin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-etc-kubernetes\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-device-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369742    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qd8xf\" (UniqueName: \"kubernetes.io/projected/d5bce780-86fb-46f1-9c39-f67622b275a9-kube-api-access-qd8xf\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-hostroot\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369801    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-daemon-config\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369825    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-multus-certs\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369853    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jq9ct\" (UniqueName: \"kubernetes.io/projected/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-kube-api-access-jq9ct\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-node-log\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369924    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.369958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-registration-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369959    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-sys-fs\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.369974    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gh5r5\" (UniqueName: \"kubernetes.io/projected/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-kube-api-access-gh5r5\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-systemd-units\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-socket-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-system-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.370468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370075    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.370669 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370563    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.370868 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.370846    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.371027 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.371000    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 12:48:24.371075 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.371035    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.371350 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.371336    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4nvpl\""
Sep 09 12:48:24.373196 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.373176    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.373360 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.373342    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5wpd4\""
Sep 09 12:48:24.373427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.373347    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.373481 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.373456    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.373877 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.373858    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 12:48:24.375631 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.375614    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 12:48:24.375907 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.375893    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.376011 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.375996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 12:48:24.376081 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.376013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-zjbzx\""
Sep 09 12:48:24.378302 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.378276    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.378397 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.378357    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:24.378612 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.378593    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-4pllp\""
Sep 09 12:48:24.378680 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.378598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 12:48:24.378680 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.378616    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 12:48:24.379099 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.379085    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 12:48:24.380832 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.380813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:24.380917 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.380863    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:24.403074 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.403056    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5289"
Sep 09 12:48:24.412370 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.412352    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w5289"
Sep 09 12:48:24.451122 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.451082    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal" event={"ID":"d936d2fc428f3884c06a358b12bfbd88","Type":"ContainerStarted","Data":"4fd9b05c0005ed47c8c44ea1f1a19d6b05ecf6961525052c7e43c4397fd4c6c9"}
Sep 09 12:48:24.452224 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.452204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal" event={"ID":"270358c4069fad120920bbf1f2675138","Type":"ContainerStarted","Data":"2837c6a95b74f0fadc42163e21e62ddbf823141167a5ece48f01784146642377"}
Sep 09 12:48:24.459602 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.459587    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 12:48:24.471201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471181    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-kubelet\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-modprobe-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.471289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471226    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-cnibin\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.471289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.471289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471258    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-bin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471273    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-etc-kubernetes\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471303    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jq9ct\" (UniqueName: \"kubernetes.io/projected/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-kube-api-access-jq9ct\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471328    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-host\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471330    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-bin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471361    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471397    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-etc-kubernetes\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-etc-selinux\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.471475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471472    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-kubelet\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qd8xf\" (UniqueName: \"kubernetes.io/projected/d5bce780-86fb-46f1-9c39-f67622b275a9-kube-api-access-qd8xf\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-daemon-config\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471560    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-multus-certs\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t9jfm\" (UniqueName: \"kubernetes.io/projected/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-kube-api-access-t9jfm\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471599    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-agent-certs\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-node-log\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471639    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-registration-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-multus-certs\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471677    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29c984e6-6c2f-4784-bb63-3b13e74b7af6-host\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471702    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-registration-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-kubernetes\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-node-log\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.471757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471745    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-systemd\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471775    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-system-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471829    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29c984e6-6c2f-4784-bb63-3b13e74b7af6-serviceca\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-system-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471848    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c016c326-ee63-4f22-9308-ab7a015c0841-host-slash\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-script-lib\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cnibin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-cni-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-socket-dir-parent\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.471978    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cnibin\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-tmp\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-daemon-config\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-socket-dir-parent\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472073    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472105    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-netd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-env-overrides\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.472404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472164    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-netd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-multus\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472200    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-run\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7j7w\" (UniqueName: \"kubernetes.io/projected/421a4317-7258-4c56-a856-4a2e63df7522-kube-api-access-p7j7w\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-kubelet\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-var-lib-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-bin\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472284    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktkhk\" (UniqueName: \"kubernetes.io/projected/29c984e6-6c2f-4784-bb63-3b13e74b7af6-kube-api-access-ktkhk\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472297    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-var-lib-cni-multus\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-kubelet\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472318    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-conf\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472332    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-cni-bin\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-var-lib-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-konnectivity-ca\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472385    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cni-binary-copy\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472409    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-script-lib\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-conf-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472468    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-multus-conf-dir\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472486    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjqqk\" (UniqueName: \"kubernetes.io/projected/c016c326-ee63-4f22-9308-ab7a015c0841-kube-api-access-hjqqk\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472525    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-slash\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-log-socket\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-slash\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472554    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-env-overrides\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-log-socket\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-config\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472662    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/333cd3fd-e195-43da-95c1-df8ba568219f-ovn-node-metrics-cert\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-k8s-cni-cncf-io\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472758    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472787    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-device-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-cni-binary-copy\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.473978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-hostroot\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472839    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-k8s-cni-cncf-io\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472850    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-dbus\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-hostroot\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472881    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-tuned\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472909    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-device-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472932    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-sys-fs\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472949    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gh5r5\" (UniqueName: \"kubernetes.io/projected/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-kube-api-access-gh5r5\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472966    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-var-lib-kubelet\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-sys-fs\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472985    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-tmp-dir\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-system-cni-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.472982    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/333cd3fd-e195-43da-95c1-df8ba568219f-ovnkube-config\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-systemd-units\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.474498 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-socket-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-kubelet-config\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473181    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-sys\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-systemd-units\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473203    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-lib-modules\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473241    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c016c326-ee63-4f22-9308-ab7a015c0841-iptables-alerter-script\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-hosts-file\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473265    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d5bce780-86fb-46f1-9c39-f67622b275a9-socket-dir\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkthj\" (UniqueName: \"kubernetes.io/projected/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-kube-api-access-gkthj\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-netns\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473338    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-host-run-netns\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysconfig\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473407    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-netns\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-systemd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-etc-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.473447    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:24.475066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473456    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473493    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-systemd\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473501    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-netns\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-etc-openvswitch\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.473535    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:24.973493478 +0000 UTC m=+3.008741382 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-ovn\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473587    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgllp\" (UniqueName: \"kubernetes.io/projected/333cd3fd-e195-43da-95c1-df8ba568219f-kube-api-access-tgllp\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473617    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-run-ovn\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.475610 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.473637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/333cd3fd-e195-43da-95c1-df8ba568219f-host-run-ovn-kubernetes\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.476791 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.476774    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/333cd3fd-e195-43da-95c1-df8ba568219f-ovn-node-metrics-cert\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.483931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.483911    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jq9ct\" (UniqueName: \"kubernetes.io/projected/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-kube-api-access-jq9ct\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.484034 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.483961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgllp\" (UniqueName: \"kubernetes.io/projected/333cd3fd-e195-43da-95c1-df8ba568219f-kube-api-access-tgllp\") pod \"ovnkube-node-qdwjh\" (UID: \"333cd3fd-e195-43da-95c1-df8ba568219f\") " pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.484233 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.484214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gh5r5\" (UniqueName: \"kubernetes.io/projected/39d9ea4d-5b06-4709-8e38-e5cbf013ac50-kube-api-access-gh5r5\") pod \"multus-v5mbc\" (UID: \"39d9ea4d-5b06-4709-8e38-e5cbf013ac50\") " pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.485376 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.485356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qd8xf\" (UniqueName: \"kubernetes.io/projected/d5bce780-86fb-46f1-9c39-f67622b275a9-kube-api-access-qd8xf\") pod \"aws-ebs-csi-driver-node-tckk7\" (UID: \"d5bce780-86fb-46f1-9c39-f67622b275a9\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.574398 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574335    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-system-cni-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574398 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574373    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-kubelet-config\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-system-cni-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574461    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-sys\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-lib-modules\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c016c326-ee63-4f22-9308-ab7a015c0841-iptables-alerter-script\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.574560 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-hosts-file\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574523    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-kubelet-config\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-tuning-conf-dir\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574557    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-sys\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkthj\" (UniqueName: \"kubernetes.io/projected/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-kube-api-access-gkthj\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-lib-modules\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574643    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysconfig\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574651    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-hosts-file\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-modprobe-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-cnibin\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysconfig\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574749    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-host\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-modprobe-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574833    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/421a4317-7258-4c56-a856-4a2e63df7522-cnibin\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574847    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t9jfm\" (UniqueName: \"kubernetes.io/projected/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-kube-api-access-t9jfm\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.574857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-host\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-agent-certs\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574899    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29c984e6-6c2f-4784-bb63-3b13e74b7af6-host\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-kubernetes\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-systemd\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29c984e6-6c2f-4784-bb63-3b13e74b7af6-serviceca\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574986    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/29c984e6-6c2f-4784-bb63-3b13e74b7af6-host\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.574997    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c016c326-ee63-4f22-9308-ab7a015c0841-host-slash\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575051    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-tmp\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575060    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-systemd\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-kubernetes\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575073    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/c016c326-ee63-4f22-9308-ab7a015c0841-iptables-alerter-script\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575076    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575120    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c016c326-ee63-4f22-9308-ab7a015c0841-host-slash\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575151    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-d\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-run\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575203    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-run\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.575625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7j7w\" (UniqueName: \"kubernetes.io/projected/421a4317-7258-4c56-a856-4a2e63df7522-kube-api-access-p7j7w\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktkhk\" (UniqueName: \"kubernetes.io/projected/29c984e6-6c2f-4784-bb63-3b13e74b7af6-kube-api-access-ktkhk\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-conf\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-konnectivity-ca\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575314    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-binary-copy\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575347    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjqqk\" (UniqueName: \"kubernetes.io/projected/c016c326-ee63-4f22-9308-ab7a015c0841-kube-api-access-hjqqk\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575449    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-dbus\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-tuned\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575504    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/29c984e6-6c2f-4784-bb63-3b13e74b7af6-serviceca\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-var-lib-kubelet\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575557    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-tmp-dir\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575597    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/abd06375-8c1e-49ec-9a04-9782206cadbc-dbus\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.575602    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:24.576471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575646    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-var-lib-kubelet\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.577163 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.575678    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:25.075656343 +0000 UTC m=+3.110904241 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:24.577163 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575776    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-sysctl-conf\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.577163 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.575886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-tmp-dir\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.577163 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.576155    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-konnectivity-ca\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.577317 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.577165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/421a4317-7258-4c56-a856-4a2e63df7522-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.577454 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.577436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/641d2aa8-d2be-4f45-abfe-5c18dde8f8fe-agent-certs\") pod \"konnectivity-agent-2f5np\" (UID: \"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe\") " pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.577547 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.577526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-tmp\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.577871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.577854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-etc-tuned\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.588961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.588942    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:24.588961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.588963    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:24.589134 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.588974    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:24.589134 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.589044    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:25.089027998 +0000 UTC m=+3.124275905 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:24.598107 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.598078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t9jfm\" (UniqueName: \"kubernetes.io/projected/0e9025e3-a6ee-4faa-9619-86ad5b1516c5-kube-api-access-t9jfm\") pod \"tuned-p6f5w\" (UID: \"0e9025e3-a6ee-4faa-9619-86ad5b1516c5\") " pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.600228 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.599766    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkthj\" (UniqueName: \"kubernetes.io/projected/853f7946-ed3b-44b7-a2d8-fc6e57cf18b0-kube-api-access-gkthj\") pod \"node-resolver-8ngjq\" (UID: \"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0\") " pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.600902 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.600879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjqqk\" (UniqueName: \"kubernetes.io/projected/c016c326-ee63-4f22-9308-ab7a015c0841-kube-api-access-hjqqk\") pod \"iptables-alerter-whwqr\" (UID: \"c016c326-ee63-4f22-9308-ab7a015c0841\") " pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.600902 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.600893    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7j7w\" (UniqueName: \"kubernetes.io/projected/421a4317-7258-4c56-a856-4a2e63df7522-kube-api-access-p7j7w\") pod \"multus-additional-cni-plugins-rjk6k\" (UID: \"421a4317-7258-4c56-a856-4a2e63df7522\") " pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.601013 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.600934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktkhk\" (UniqueName: \"kubernetes.io/projected/29c984e6-6c2f-4784-bb63-3b13e74b7af6-kube-api-access-ktkhk\") pod \"node-ca-rc72g\" (UID: \"29c984e6-6c2f-4784-bb63-3b13e74b7af6\") " pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.664393 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.664360    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:24.671032 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.670971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7"
Sep 09 12:48:24.679343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.678551    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-v5mbc"
Sep 09 12:48:24.682870 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:24.682847    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5bce780_86fb_46f1_9c39_f67622b275a9.slice/crio-3e0fa882d6f49057d50724925ca6bd264c402dddd06c43da3be511e73fbbcc52 WatchSource:0}: Error finding container 3e0fa882d6f49057d50724925ca6bd264c402dddd06c43da3be511e73fbbcc52: Status 404 returned error can't find the container with id 3e0fa882d6f49057d50724925ca6bd264c402dddd06c43da3be511e73fbbcc52
Sep 09 12:48:24.683764 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.683740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:24.688832 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:24.688614    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39d9ea4d_5b06_4709_8e38_e5cbf013ac50.slice/crio-9a319c9e52d6681cebeb4825b283e36d157d8f52ecc1d1dd3d208e8e87de82c9 WatchSource:0}: Error finding container 9a319c9e52d6681cebeb4825b283e36d157d8f52ecc1d1dd3d208e8e87de82c9: Status 404 returned error can't find the container with id 9a319c9e52d6681cebeb4825b283e36d157d8f52ecc1d1dd3d208e8e87de82c9
Sep 09 12:48:24.689469 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.689448    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-p6f5w"
Sep 09 12:48:24.696601 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.695602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-whwqr"
Sep 09 12:48:24.706615 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.706580    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-8ngjq"
Sep 09 12:48:24.708650 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.708479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-rjk6k"
Sep 09 12:48:24.712021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.712004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-rc72g"
Sep 09 12:48:24.720801 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:24.720771    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod853f7946_ed3b_44b7_a2d8_fc6e57cf18b0.slice/crio-67d6ffbe51071dd542eeb4c5d9c7c1f55b5ce2e5d20974d08377c7aef3486875 WatchSource:0}: Error finding container 67d6ffbe51071dd542eeb4c5d9c7c1f55b5ce2e5d20974d08377c7aef3486875: Status 404 returned error can't find the container with id 67d6ffbe51071dd542eeb4c5d9c7c1f55b5ce2e5d20974d08377c7aef3486875
Sep 09 12:48:24.725598 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:24.725570    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod421a4317_7258_4c56_a856_4a2e63df7522.slice/crio-9865ba5e9c002a2c52f76c0353a0b4d544c1c72fb4d0acc66378311acc350603 WatchSource:0}: Error finding container 9865ba5e9c002a2c52f76c0353a0b4d544c1c72fb4d0acc66378311acc350603: Status 404 returned error can't find the container with id 9865ba5e9c002a2c52f76c0353a0b4d544c1c72fb4d0acc66378311acc350603
Sep 09 12:48:24.727358 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:24.727333    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29c984e6_6c2f_4784_bb63_3b13e74b7af6.slice/crio-abe0caaca590e048e371d2e4fa9f4383794fae73be2fcf2cb4e99a5aeda79eed WatchSource:0}: Error finding container abe0caaca590e048e371d2e4fa9f4383794fae73be2fcf2cb4e99a5aeda79eed: Status 404 returned error can't find the container with id abe0caaca590e048e371d2e4fa9f4383794fae73be2fcf2cb4e99a5aeda79eed
Sep 09 12:48:24.979876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:24.979792    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:24.980040 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.979914    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:24.980040 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:24.979987    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:25.979963913 +0000 UTC m=+4.015211802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:25.080290 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.080254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:25.080504 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.080448    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:25.080504 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.080497    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:26.080483405 +0000 UTC m=+4.115731292 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:25.180996 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.180945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:25.181185 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.181116    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:25.181185 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.181140    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:25.181185 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.181153    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:25.181402 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.181212    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:26.181193598 +0000 UTC m=+4.216441506 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:25.415745 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.415648    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:43:25 +0000 UTC" deadline="2028-04-17 19:08:33.13194745 +0000 UTC"
Sep 09 12:48:25.415745 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.415699    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14070h20m7.716251805s"
Sep 09 12:48:25.455971 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.455935    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerStarted","Data":"9865ba5e9c002a2c52f76c0353a0b4d544c1c72fb4d0acc66378311acc350603"}
Sep 09 12:48:25.457260 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.457232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p6f5w" event={"ID":"0e9025e3-a6ee-4faa-9619-86ad5b1516c5","Type":"ContainerStarted","Data":"537c51db187571a6b03831e1d3910f2db9bccfc801ddcf74320ae16152e97fd1"}
Sep 09 12:48:25.458624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.458598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2f5np" event={"ID":"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe","Type":"ContainerStarted","Data":"04ab3cc6b802580f2b2bea8654060d3d9101435067e08b307585c05528846e7c"}
Sep 09 12:48:25.459876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.459851    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5mbc" event={"ID":"39d9ea4d-5b06-4709-8e38-e5cbf013ac50","Type":"ContainerStarted","Data":"9a319c9e52d6681cebeb4825b283e36d157d8f52ecc1d1dd3d208e8e87de82c9"}
Sep 09 12:48:25.461059 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.461035    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7" event={"ID":"d5bce780-86fb-46f1-9c39-f67622b275a9","Type":"ContainerStarted","Data":"3e0fa882d6f49057d50724925ca6bd264c402dddd06c43da3be511e73fbbcc52"}
Sep 09 12:48:25.462341 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.462312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"ee51dd3d51ac25ab275f730818291a9a74ba3dfb7d4795bfdfb23075cec24dd4"}
Sep 09 12:48:25.463581 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.463559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rc72g" event={"ID":"29c984e6-6c2f-4784-bb63-3b13e74b7af6","Type":"ContainerStarted","Data":"abe0caaca590e048e371d2e4fa9f4383794fae73be2fcf2cb4e99a5aeda79eed"}
Sep 09 12:48:25.464826 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.464804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8ngjq" event={"ID":"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0","Type":"ContainerStarted","Data":"67d6ffbe51071dd542eeb4c5d9c7c1f55b5ce2e5d20974d08377c7aef3486875"}
Sep 09 12:48:25.466039 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.466019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-whwqr" event={"ID":"c016c326-ee63-4f22-9308-ab7a015c0841","Type":"ContainerStarted","Data":"010bf74f6ad38a5fcd458e228423cb5af7e006c4017269cc6743a256d0fadbc3"}
Sep 09 12:48:25.788609 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.788206    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 12:48:25.988066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:25.988026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:25.988275 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.988197    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:25.988275 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:25.988268    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:27.988247983 +0000 UTC m=+6.023495875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:26.089397 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.089315    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:26.089583 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.089483    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:26.089583 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.089567    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:28.089532683 +0000 UTC m=+6.124780574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:26.189848 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.189777    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:26.190055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.190037    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:26.190132 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.190066    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:26.190132 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.190080    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:26.190243 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.190141    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:28.190123042 +0000 UTC m=+6.225370936 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:26.417586 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.417487    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 12:43:25 +0000 UTC" deadline="2028-02-23 03:57:48.268249996 +0000 UTC"
Sep 09 12:48:26.417586 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.417528    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12759h9m21.850726784s"
Sep 09 12:48:26.449473 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.449443    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:26.449662 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.449595    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:26.450041 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.450018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:26.450158 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.450136    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:26.451034 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:26.450216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:26.451034 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:26.450364    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:27.479492 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:27.479430    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal" event={"ID":"270358c4069fad120920bbf1f2675138","Type":"ContainerStarted","Data":"619bad9349c86d1b201338f47da7918f7c4c23e75eeb55c4122084ab122fb84f"}
Sep 09 12:48:28.007199 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.007108    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:28.007376 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.007310    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:28.007439 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.007392    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:32.007370318 +0000 UTC m=+10.042618205 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:28.108265 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.108226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:28.108459 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.108374    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:28.108459 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.108444    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:32.10842518 +0000 UTC m=+10.143673082 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:28.208979 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.208929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:28.209217 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.209172    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:28.209217 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.209197    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:28.209217 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.209211    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:28.209433 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.209275    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:32.209256064 +0000 UTC m=+10.244503972 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:28.447778 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.447695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:28.447946 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.447840    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:28.448239 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.448219    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:28.448344 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.448320    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:28.448390 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.448381    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:28.448470 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:28.448452    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:28.494316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.494200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal" event={"ID":"d936d2fc428f3884c06a358b12bfbd88","Type":"ContainerStarted","Data":"8d1419f17f2cf2a327528fd785b60e39b42c09d5205098ce9bc83848b11f1a9d"}
Sep 09 12:48:28.510296 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:28.509880    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-1.ec2.internal" podStartSLOduration=4.509859757 podStartE2EDuration="4.509859757s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:27.497735287 +0000 UTC m=+5.532983195" watchObservedRunningTime="2026-09-09 12:48:28.509859757 +0000 UTC m=+6.545107670"
Sep 09 12:48:29.500042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:29.499858    2580 generic.go:356] "Generic (PLEG): container finished" podID="d936d2fc428f3884c06a358b12bfbd88" containerID="8d1419f17f2cf2a327528fd785b60e39b42c09d5205098ce9bc83848b11f1a9d" exitCode=0
Sep 09 12:48:29.500042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:29.499915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal" event={"ID":"d936d2fc428f3884c06a358b12bfbd88","Type":"ContainerDied","Data":"8d1419f17f2cf2a327528fd785b60e39b42c09d5205098ce9bc83848b11f1a9d"}
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:30.450004    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:30.450130    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:30.450131    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:30.450205    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:30.450269    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:30.450492 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:30.450376    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:32.042900 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.042869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:32.043276 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.042986    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:32.043276 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.043039    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:40.043023643 +0000 UTC m=+18.078271531 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:32.144036 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.143930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:32.144209 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.144106    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:32.144209 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.144181    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:40.144159725 +0000 UTC m=+18.179407620 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:32.245393 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.245349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:32.245589 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.245560    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:32.245589 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.245585    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:32.245697 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.245599    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:32.245697 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.245659    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:40.245640212 +0000 UTC m=+18.280888103 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:32.447047 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.446480    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:32.447047 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.446631    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:32.447047 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.446948    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:32.447321 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.447051    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:32.447321 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:32.447227    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:32.448397 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:32.448358    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:34.446407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:34.446125    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:34.446407 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:34.446285    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:34.446407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:34.446128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:34.446407 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:34.446383    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:34.446407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:34.446123    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:34.447008 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:34.446468    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:36.446664 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:36.446636    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:36.446965 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:36.446643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:36.446965 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:36.446746    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:36.446965 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:36.446808    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:36.446965 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:36.446648    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:36.446965 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:36.446901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:38.446117 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:38.446086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:38.446117 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:38.446104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:38.446617 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:38.446092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:38.446617 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:38.446199    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:38.446617 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:38.446313    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:38.446617 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:38.446403    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:40.104060 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.104027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:40.104498 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.104181    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:40.104498 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.104249    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:56.104229244 +0000 UTC m=+34.139477200 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:40.205275 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.205242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:40.205421 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.205353    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:40.205421 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.205404    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:48:56.205391667 +0000 UTC m=+34.240639552 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:40.305990 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.305957    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:40.306158 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.306112    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:40.306158 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.306127    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:40.306158 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.306136    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:40.306308 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.306186    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:48:56.306169415 +0000 UTC m=+34.341417316 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:40.446743 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.446663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:40.446743 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.446695    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:40.446939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:40.446663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:40.446939 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.446793    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:40.446939 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.446851    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:40.447069 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:40.446953    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:42.446809 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:42.446780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:42.446809 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:42.446810    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:42.447292 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:42.446901    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:42.447292 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:42.446933    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:42.447292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:42.446978    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:42.447292 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:42.447050    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:44.446556 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.446376    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:44.447203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.446375    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:44.447203 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:44.446629    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:44.447203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.446435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:44.447203 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:44.446747    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:44.447203 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:44.446844    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:44.530198 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.530136    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-rc72g" event={"ID":"29c984e6-6c2f-4784-bb63-3b13e74b7af6","Type":"ContainerStarted","Data":"052a6db43c875a9cbba544ceb6950384e5e7988a08bcea4b0dff912fd85f5c43"}
Sep 09 12:48:44.531581 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.531559    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-8ngjq" event={"ID":"853f7946-ed3b-44b7-a2d8-fc6e57cf18b0","Type":"ContainerStarted","Data":"221836d6d25f402b8068e91fc6e3b627f8c17ccff3b4df7e6f649b6be62b4937"}
Sep 09 12:48:44.533055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.533036    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="0dbeb7d32e1bf25d050a06e45f17e00e0fe19c58d4d5aa3c66646c04e443d8eb" exitCode=0
Sep 09 12:48:44.533138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.533101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"0dbeb7d32e1bf25d050a06e45f17e00e0fe19c58d4d5aa3c66646c04e443d8eb"}
Sep 09 12:48:44.534659 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.534637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-p6f5w" event={"ID":"0e9025e3-a6ee-4faa-9619-86ad5b1516c5","Type":"ContainerStarted","Data":"8ce6d5a82a86fcd270d486dded7a6cf4bdeeb4596714fb47b7d2309a501288f2"}
Sep 09 12:48:44.536075 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.536058    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-2f5np" event={"ID":"641d2aa8-d2be-4f45-abfe-5c18dde8f8fe","Type":"ContainerStarted","Data":"edefeee9d4f5bdf77ea2fa757296efef33d3cba9a5d5b1c7891cafd7050c9f84"}
Sep 09 12:48:44.537478 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.537458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-v5mbc" event={"ID":"39d9ea4d-5b06-4709-8e38-e5cbf013ac50","Type":"ContainerStarted","Data":"c3ab076854b1be5439ca72ee2c6dc80502f0f63c2369f244d5808b2c8bf86185"}
Sep 09 12:48:44.538908 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.538888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7" event={"ID":"d5bce780-86fb-46f1-9c39-f67622b275a9","Type":"ContainerStarted","Data":"13b916e503e0158063703ab1ef33e51d722dcda684e28984d76b10c1127f2c1b"}
Sep 09 12:48:44.541544 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541522    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"bd9274ba1dec7ad9f98dd1528b68a37c1050e2c0e52ab275bcc4e38f413aad61"}
Sep 09 12:48:44.541621 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"8019e68fc9350ce86b86460be55a27d5599136f1cb27a6a008d76efd03799c6a"}
Sep 09 12:48:44.541668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541626    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"9ee199ef883156d932cb6b7dec2ed5345362614f700a0f0924cc7f56a568dace"}
Sep 09 12:48:44.541668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541655    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"df48eae30e7432e8fe410fbd95fe33763dd246f2c8fd0b12a9e3b8bf621e753f"}
Sep 09 12:48:44.541743 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541687    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"ec3c8a4b8dfe9f2165d11b0f6d0c6e4f8d1b9899897b838bd9a62fef333eda17"}
Sep 09 12:48:44.541743 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.541718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"fdb6d216a3628d6457b7f7e35e6d9a364e87c2686c0a049d5ae904bf0f63b18f"}
Sep 09 12:48:44.543251 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.543231    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal" event={"ID":"d936d2fc428f3884c06a358b12bfbd88","Type":"ContainerStarted","Data":"8680ae6ea3f5b4d8379664f666b63bb99d2f95f942bd0cb7c39f2fb816f7a996"}
Sep 09 12:48:44.544429 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.544396    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-rc72g" podStartSLOduration=9.794867876 podStartE2EDuration="21.544382634s" totalImagesPullingTime="11.749514758s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.544072677 +0000 UTC m=+22.579320584" watchObservedRunningTime="2026-09-09 12:48:44.544382634 +0000 UTC m=+22.579630542"
Sep 09 12:48:44.559094 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.559060    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-1.ec2.internal" podStartSLOduration=20.559049414 podStartE2EDuration="20.559049414s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.558841122 +0000 UTC m=+22.594089029" watchObservedRunningTime="2026-09-09 12:48:44.559049414 +0000 UTC m=+22.594297321"
Sep 09 12:48:44.596145 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.596110    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-v5mbc" podStartSLOduration=2.524141524 podStartE2EDuration="21.59610012s" totalImagesPullingTime="19.071958596s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.578877503 +0000 UTC m=+22.614125411" watchObservedRunningTime="2026-09-09 12:48:44.59610012 +0000 UTC m=+22.631348026"
Sep 09 12:48:44.596257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.596239    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-p6f5w" podStartSLOduration=2.579867148 podStartE2EDuration="21.596235703s" totalImagesPullingTime="19.016368555s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.596210069 +0000 UTC m=+22.631457954" watchObservedRunningTime="2026-09-09 12:48:44.596235703 +0000 UTC m=+22.631483609"
Sep 09 12:48:44.613755 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.613721    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-8ngjq" podStartSLOduration=2.8774465879999997 podStartE2EDuration="21.613710027s" totalImagesPullingTime="18.736263439s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.61325312 +0000 UTC m=+22.648501027" watchObservedRunningTime="2026-09-09 12:48:44.613710027 +0000 UTC m=+22.648957933"
Sep 09 12:48:44.666395 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:44.666358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-2f5np" podStartSLOduration=2.670562334 podStartE2EDuration="21.666347017s" totalImagesPullingTime="18.995784683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:44.66591328 +0000 UTC m=+22.701161198" watchObservedRunningTime="2026-09-09 12:48:44.666347017 +0000 UTC m=+22.701594924"
Sep 09 12:48:45.223629 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.223608    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 12:48:45.427784 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.427630    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T12:48:45.223624531Z","UUID":"9e6b551a-465b-4676-af2c-f46785cff39a","Handler":null,"Name":"","Endpoint":""}
Sep 09 12:48:45.429537 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.429495    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 12:48:45.429710 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.429546    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 12:48:45.547472 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.547434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7" event={"ID":"d5bce780-86fb-46f1-9c39-f67622b275a9","Type":"ContainerStarted","Data":"779b967fb8608f1e462a16b57909de37a0ecd4e7449cfd473d525e8879dec672"}
Sep 09 12:48:45.549162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.549138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-whwqr" event={"ID":"c016c326-ee63-4f22-9308-ab7a015c0841","Type":"ContainerStarted","Data":"994c1f2aebcc025a0cb0517bc92e37813ca8a0b392c7282d4981dae3753781b2"}
Sep 09 12:48:45.563368 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.563319    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-whwqr" podStartSLOduration=3.814775281 podStartE2EDuration="22.563302042s" totalImagesPullingTime="18.748526761s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:45.563289608 +0000 UTC m=+23.598537514" watchObservedRunningTime="2026-09-09 12:48:45.563302042 +0000 UTC m=+23.598549949"
Sep 09 12:48:45.695850 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.695787    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:45.696292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:45.696250    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:46.446976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.446863    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:46.447245 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:46.447029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:46.447383 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.447359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:46.447495 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:46.447477    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:46.447593 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.447555    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:46.447671 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:46.447649    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:46.557255 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.557219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7" event={"ID":"d5bce780-86fb-46f1-9c39-f67622b275a9","Type":"ContainerStarted","Data":"c02cd383cc80f7aefe576504dcc3782dfa8f95bf8cb880d523a4e26c296caa46"}
Sep 09 12:48:46.562039 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.562008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"bf58f5313570edcb6441fe451dba9cc18a75a55552515877e28655e5d2ec0dd5"}
Sep 09 12:48:46.562340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.562321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:46.562847 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.562817    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-2f5np"
Sep 09 12:48:46.572390 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:46.572330    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tckk7" podStartSLOduration=2.478896391 podStartE2EDuration="23.57231888s" totalImagesPullingTime="21.093422489s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:46.572294641 +0000 UTC m=+24.607542550" watchObservedRunningTime="2026-09-09 12:48:46.57231888 +0000 UTC m=+24.607566786"
Sep 09 12:48:48.446453 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:48.446421    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:48.446968 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:48.446433    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:48.446968 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:48.446562    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:48.446968 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:48.446591    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:48.446968 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:48.446714    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:48.446968 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:48.446801    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:50.446703 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.446672    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:50.447152 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:50.446773    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:50.447152 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.446677    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:50.447152 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.446816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:50.447152 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:50.446857    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:50.447152 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:50.446951    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:50.571434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.571403    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="a0e659cdb43a3e633395c76f07c501214d503be32668a47da84776c81ac7959f" exitCode=0
Sep 09 12:48:50.571599 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.571489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"a0e659cdb43a3e633395c76f07c501214d503be32668a47da84776c81ac7959f"}
Sep 09 12:48:50.575378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.575355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" event={"ID":"333cd3fd-e195-43da-95c1-df8ba568219f","Type":"ContainerStarted","Data":"af15f2a96287a1a887a6b5c812f39477934e6dceda9fbc4806bb82316361f555"}
Sep 09 12:48:50.575642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.575613    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:50.575729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.575698    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:50.575799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.575747    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:50.590296 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.590277    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:50.590380 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.590364    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:48:50.614762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:50.614720    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh" podStartSLOduration=8.500384755 podStartE2EDuration="27.614708017s" totalImagesPullingTime="19.114323262s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:48:50.613439398 +0000 UTC m=+28.648687302" watchObservedRunningTime="2026-09-09 12:48:50.614708017 +0000 UTC m=+28.649955924"
Sep 09 12:48:51.527860 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.527829    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fj9l6"]
Sep 09 12:48:51.528427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.527939    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:51.528427 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:51.528048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:51.531326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.531301    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7pfr"]
Sep 09 12:48:51.531438 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.531385    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:51.531481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:51.531467    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:51.531955 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.531934    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkb9w"]
Sep 09 12:48:51.532030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:51.532010    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:51.532113 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:51.532093    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:52.581169 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:52.581139    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="d9b859e4c56fd179a42fbe517d9592c5e94bba5443ac67348647be84709e7656" exitCode=0
Sep 09 12:48:52.581602 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:52.581269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"d9b859e4c56fd179a42fbe517d9592c5e94bba5443ac67348647be84709e7656"}
Sep 09 12:48:53.446529 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:53.446359    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:53.446645 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:53.446410    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:53.446645 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:53.446595    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:53.446707 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:53.446679    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:53.446707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:53.446424    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:53.446776 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:53.446763    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:54.586897 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:54.586860    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="bde9cd3fa674be0b46a140ebb2f75c6b1c63f9efdbb20844a2aad3d645cad150" exitCode=0
Sep 09 12:48:54.587255 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:54.586917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"bde9cd3fa674be0b46a140ebb2f75c6b1c63f9efdbb20844a2aad3d645cad150"}
Sep 09 12:48:55.447067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:55.447035    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:55.447340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:55.447137    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:55.447340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:55.447147    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-hkb9w" podUID="43acb564-ce77-4108-a5be-23fe3d0ea28e"
Sep 09 12:48:55.447340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:55.447164    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:55.447340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:55.447247    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-q7pfr" podUID="e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799"
Sep 09 12:48:55.447340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:55.447315    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-fj9l6" podUID="abd06375-8c1e-49ec-9a04-9782206cadbc"
Sep 09 12:48:56.119775 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.119742    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:56.120190 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.119870    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:56.120190 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.119923    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs podName:e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.119909982 +0000 UTC m=+66.155157866 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs") pod "network-metrics-daemon-q7pfr" (UID: "e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 12:48:56.220641 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.220609    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:56.220808 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.220733    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:56.220808 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.220783    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret podName:abd06375-8c1e-49ec-9a04-9782206cadbc nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.220770832 +0000 UTC m=+66.256018717 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret") pod "global-pull-secret-syncer-fj9l6" (UID: "abd06375-8c1e-49ec-9a04-9782206cadbc") : object "kube-system"/"original-pull-secret" not registered
Sep 09 12:48:56.321242 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.321216    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:56.321347 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.321335    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 12:48:56.321386 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.321353    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 12:48:56.321386 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.321361    2580 projected.go:196] Error preparing data for projected volume kube-api-access-bq4hn for pod openshift-network-diagnostics/network-check-target-hkb9w: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:56.321445 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:56.321403    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn podName:43acb564-ce77-4108-a5be-23fe3d0ea28e nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.32139089 +0000 UTC m=+66.356638774 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bq4hn" (UniqueName: "kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn") pod "network-check-target-hkb9w" (UID: "43acb564-ce77-4108-a5be-23fe3d0ea28e") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 12:48:56.707871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.707840    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-137-1.ec2.internal" event="NodeReady"
Sep 09 12:48:56.708032 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.707947    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 12:48:56.746541 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.746495    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"]
Sep 09 12:48:56.750217 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.750196    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:48:56.750384 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.750369    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"
Sep 09 12:48:56.754203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.753914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.754203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.753970    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.754203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.754085    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-6pxtz\""
Sep 09 12:48:56.754731 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.754701    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"]
Sep 09 12:48:56.754929 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.754909    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.759491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.757397    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qkhdl\""
Sep 09 12:48:56.759491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.757674    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 12:48:56.759491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.758083    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 12:48:56.759491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.758429    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 12:48:56.760927 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.760179    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"]
Sep 09 12:48:56.762217 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.762196    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:56.764766 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.764748    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 12:48:56.765666 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.765647    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-b548b666-qdt2z"]
Sep 09 12:48:56.765859 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.765844    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:56.766448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.766432    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.766577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.766560    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.766652 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.766524    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 09 12:48:56.766703 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.766569    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-lqqrc\""
Sep 09 12:48:56.766752 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.766496    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 09 12:48:56.769113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.769097    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cg7vj"]
Sep 09 12:48:56.769684 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.769664    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.772876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.772850    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.772940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.772879    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.773166 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.773057    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Sep 09 12:48:56.773166 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.773116    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h4fmd\""
Sep 09 12:48:56.773398 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.773378    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Sep 09 12:48:56.774137 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.774121    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"]
Sep 09 12:48:56.774388 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.774370    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:56.776907 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.776890    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2chm\""
Sep 09 12:48:56.777027 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.777011    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 12:48:56.777095 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.777033    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.777169 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.776941    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.777278 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.776979    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.777340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.776897    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 12:48:56.777416 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.777400    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 12:48:56.777811 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.777794    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 12:48:56.778290 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.778101    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.778496 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.778102    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 12:48:56.778496 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.778190    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 12:48:56.778642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.778613    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-cxbsd\""
Sep 09 12:48:56.780822 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.780702    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"]
Sep 09 12:48:56.782408 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.781316    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:56.783968 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.783952    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.784082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784066    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.784274 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784258    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 12:48:56.784660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 12:48:56.784660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784448    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-74gwz\""
Sep 09 12:48:56.784905 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784891    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rp5wd"]
Sep 09 12:48:56.785070 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.784979    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:56.789647 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.789534    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"]
Sep 09 12:48:56.789647 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.789593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"]
Sep 09 12:48:56.789647 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.789616    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"]
Sep 09 12:48:56.789981 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.789653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nmtv2"]
Sep 09 12:48:56.791028 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.791009    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.792595 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.792578    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g6xt9\""
Sep 09 12:48:56.792992 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.792975    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Sep 09 12:48:56.793417 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.793395    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.793640 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.793619    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:48:56.793717 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.793661    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.793779 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.793734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2"
Sep 09 12:48:56.793864 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.793665    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2l9tv"]
Sep 09 12:48:56.795890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.794861    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.795890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.795145    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4d6mx\""
Sep 09 12:48:56.795890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.795156    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.795890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.795321    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 09 12:48:56.795890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.795410    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 09 12:48:56.796618 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.796550    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 12:48:56.797754 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.796767    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.797754 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.797450    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-xtvcx\""
Sep 09 12:48:56.797754 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.797452    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.798594 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.798578    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"]
Sep 09 12:48:56.798805 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.798792    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:56.798950 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.798928    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fj4kt"]
Sep 09 12:48:56.802003 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.801981    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 12:48:56.802082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.802034    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 12:48:56.802132 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.802092    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-4hv86\""
Sep 09 12:48:56.802330 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.802280    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 12:48:56.802494 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.802479    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 09 12:48:56.803584 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.803566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"]
Sep 09 12:48:56.803678 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.803606    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cg7vj"]
Sep 09 12:48:56.803678 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.803628    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-zr2l7"]
Sep 09 12:48:56.803796 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.803696    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:56.806117 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.806091    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 09 12:48:56.806619 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.806599    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 09 12:48:56.806760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.806743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sqpbh\""
Sep 09 12:48:56.807020 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807002    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nmtv2"]
Sep 09 12:48:56.807101 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807034    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rp5wd"]
Sep 09 12:48:56.807101 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807052    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2l9tv"]
Sep 09 12:48:56.807101 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807067    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b548b666-qdt2z"]
Sep 09 12:48:56.807101 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807080    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zr2l7"]
Sep 09 12:48:56.807101 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807101    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fj4kt"]
Sep 09 12:48:56.807358 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.807178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:56.809148 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.809133    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 12:48:56.809335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.809192    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 12:48:56.809415 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.809391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmjpj\""
Sep 09 12:48:56.825034 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.825010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbg7p\" (UniqueName: \"kubernetes.io/projected/6607ac44-3eff-4dbc-982f-8c3902f790ee-kube-api-access-bbg7p\") pod \"volume-data-source-validator-695c895bd6-z9s54\" (UID: \"6607ac44-3eff-4dbc-982f-8c3902f790ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"
Sep 09 12:48:56.897819 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.897765    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:56.925719 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/6153e58f-438f-44f4-a984-421cae427aea-kube-api-access-tbmsb\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:56.925814 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbg7p\" (UniqueName: \"kubernetes.io/projected/6607ac44-3eff-4dbc-982f-8c3902f790ee-kube-api-access-bbg7p\") pod \"volume-data-source-validator-695c895bd6-z9s54\" (UID: \"6607ac44-3eff-4dbc-982f-8c3902f790ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"
Sep 09 12:48:56.925814 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.925932 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m2fb7\" (UniqueName: \"kubernetes.io/projected/22caec24-941d-43d1-b74a-e513d530c79e-kube-api-access-m2fb7\") pod \"network-check-source-685456655f-nmtv2\" (UID: \"22caec24-941d-43d1-b74a-e513d530c79e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2"
Sep 09 12:48:56.925932 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5rc2\" (UniqueName: \"kubernetes.io/projected/7f392a16-e3e2-4bb1-b745-2575ec7debe7-kube-api-access-g5rc2\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:56.926038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925934    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qt8h6\" (UniqueName: \"kubernetes.io/projected/45fa2142-a4b5-405b-80a4-27438f6e9080-kube-api-access-qt8h6\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:56.926038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.925956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:56.926038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926000    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq52l\" (UniqueName: \"kubernetes.io/projected/66e4f7f3-6970-4189-b923-865fbac766d2-kube-api-access-vq52l\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.926187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926069    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-config\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:56.926187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926090    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f92c179-25c7-4c7a-89fb-286028524d9f-config\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:56.926187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5613ea48-561a-49c6-84da-94fa313b1668-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:56.926187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-snapshots\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926197    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926230    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f92c179-25c7-4c7a-89fb-286028524d9f-serving-cert\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926259    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-75s6b\" (UniqueName: \"kubernetes.io/projected/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-kube-api-access-75s6b\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k5prd\" (UniqueName: \"kubernetes.io/projected/5613ea48-561a-49c6-84da-94fa313b1668-kube-api-access-k5prd\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.926373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926372    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42bd9e84-8866-4e8c-a958-a96cd635510b-serving-cert\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6153e58f-438f-44f4-a984-421cae427aea-cert\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa2142-a4b5-405b-80a4-27438f6e9080-config-volume\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926471    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-trusted-ca\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926492    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6jwl\" (UniqueName: \"kubernetes.io/projected/5f92c179-25c7-4c7a-89fb-286028524d9f-kube-api-access-h6jwl\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926527    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5613ea48-561a-49c6-84da-94fa313b1668-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926550    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926575    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926596    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-default-certificate\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.926675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926641    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-tmp\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-stats-auth\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926784    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfmrp\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926902    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926927    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tknt4\" (UniqueName: \"kubernetes.io/projected/42bd9e84-8866-4e8c-a958-a96cd635510b-kube-api-access-tknt4\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb4jr\" (UniqueName: \"kubernetes.io/projected/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-kube-api-access-pb4jr\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.926990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66e4f7f3-6970-4189-b923-865fbac766d2-serving-cert\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.927006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f392a16-e3e2-4bb1-b745-2575ec7debe7-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.927020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cad0a6c1-e143-45f1-a7f3-08826bc0c048-nginx-conf\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.927041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa2142-a4b5-405b-80a4-27438f6e9080-tmp-dir\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.927070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.927187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.927099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:56.941264 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:56.941241    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbg7p\" (UniqueName: \"kubernetes.io/projected/6607ac44-3eff-4dbc-982f-8c3902f790ee-kube-api-access-bbg7p\") pod \"volume-data-source-validator-695c895bd6-z9s54\" (UID: \"6607ac44-3eff-4dbc-982f-8c3902f790ee\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"
Sep 09 12:48:57.008153 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.008123    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:57.028065 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f92c179-25c7-4c7a-89fb-286028524d9f-serving-cert\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.028178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028078    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-75s6b\" (UniqueName: \"kubernetes.io/projected/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-kube-api-access-75s6b\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.028178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028104    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k5prd\" (UniqueName: \"kubernetes.io/projected/5613ea48-561a-49c6-84da-94fa313b1668-kube-api-access-k5prd\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.028178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.028178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42bd9e84-8866-4e8c-a958-a96cd635510b-serving-cert\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6153e58f-438f-44f4-a984-421cae427aea-cert\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028226    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa2142-a4b5-405b-80a4-27438f6e9080-config-volume\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028263    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-trusted-ca\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.028275    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.028291    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:48:57.028386 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.028342    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.528323651 +0000 UTC m=+35.563571553 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028288    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6jwl\" (UniqueName: \"kubernetes.io/projected/5f92c179-25c7-4c7a-89fb-286028524d9f-kube-api-access-h6jwl\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028573    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5613ea48-561a-49c6-84da-94fa313b1668-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028630    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028661    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028724    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-default-certificate\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.028762 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028782    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-tmp\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.028788    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-stats-auth\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028831    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.028849    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.528831811 +0000 UTC m=+35.564079709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028891    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfmrp\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.028998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tknt4\" (UniqueName: \"kubernetes.io/projected/42bd9e84-8866-4e8c-a958-a96cd635510b-kube-api-access-tknt4\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb4jr\" (UniqueName: \"kubernetes.io/projected/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-kube-api-access-pb4jr\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66e4f7f3-6970-4189-b923-865fbac766d2-serving-cert\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029095    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f392a16-e3e2-4bb1-b745-2575ec7debe7-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cad0a6c1-e143-45f1-a7f3-08826bc0c048-nginx-conf\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029135    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa2142-a4b5-405b-80a4-27438f6e9080-tmp-dir\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029178    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.029216    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.529202935 +0000 UTC m=+35.564450839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029224    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-trusted-ca\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/6153e58f-438f-44f4-a984-421cae427aea-kube-api-access-tbmsb\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m2fb7\" (UniqueName: \"kubernetes.io/projected/22caec24-941d-43d1-b74a-e513d530c79e-kube-api-access-m2fb7\") pod \"network-check-source-685456655f-nmtv2\" (UID: \"22caec24-941d-43d1-b74a-e513d530c79e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5rc2\" (UniqueName: \"kubernetes.io/projected/7f392a16-e3e2-4bb1-b745-2575ec7debe7-kube-api-access-g5rc2\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029387    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qt8h6\" (UniqueName: \"kubernetes.io/projected/45fa2142-a4b5-405b-80a4-27438f6e9080-kube-api-access-qt8h6\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029415    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/45fa2142-a4b5-405b-80a4-27438f6e9080-tmp-dir\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029427    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq52l\" (UniqueName: \"kubernetes.io/projected/66e4f7f3-6970-4189-b923-865fbac766d2-kube-api-access-vq52l\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.029493    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-config\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.029941 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.029546    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.529534464 +0000 UTC m=+35.564782364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:57.030713 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029563    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f92c179-25c7-4c7a-89fb-286028524d9f-config\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.030713 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5613ea48-561a-49c6-84da-94fa313b1668-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.030713 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-snapshots\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.030713 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.029644    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.031557 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.031538    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:48:57.031810 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.031793    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.531773976 +0000 UTC m=+35.567021878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:48:57.032024 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.031998    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/42bd9e84-8866-4e8c-a958-a96cd635510b-config\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.033006 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.032919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.033111 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.033050    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5f92c179-25c7-4c7a-89fb-286028524d9f-config\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.033586 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.033566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/5613ea48-561a-49c6-84da-94fa313b1668-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.034133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/45fa2142-a4b5-405b-80a4-27438f6e9080-config-volume\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.034183    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-snapshots\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.034614    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.034711    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/42bd9e84-8866-4e8c-a958-a96cd635510b-serving-cert\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.034756    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.534741859 +0000 UTC m=+35.569989760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.034858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5613ea48-561a-49c6-84da-94fa313b1668-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.034895    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:48:57.035340 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.034945    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:57.53492835 +0000 UTC m=+35.570176235 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:48:57.035752 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.035608    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.035962 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.035931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/66e4f7f3-6970-4189-b923-865fbac766d2-tmp\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.036050 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.035935    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66e4f7f3-6970-4189-b923-865fbac766d2-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.036050 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.036040    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.037208 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.037187    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/cad0a6c1-e143-45f1-a7f3-08826bc0c048-nginx-conf\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:57.037360 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.037338    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7f392a16-e3e2-4bb1-b745-2575ec7debe7-telemetry-config\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.040739 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.040715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-default-certificate\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.041445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.040905    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5f92c179-25c7-4c7a-89fb-286028524d9f-serving-cert\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.041445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.041191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/6153e58f-438f-44f4-a984-421cae427aea-cert\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:57.041445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.041369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-stats-auth\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.041544    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.041754    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.042255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-75s6b\" (UniqueName: \"kubernetes.io/projected/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-kube-api-access-75s6b\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.042328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5rc2\" (UniqueName: \"kubernetes.io/projected/7f392a16-e3e2-4bb1-b745-2575ec7debe7-kube-api-access-g5rc2\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.042410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq52l\" (UniqueName: \"kubernetes.io/projected/66e4f7f3-6970-4189-b923-865fbac766d2-kube-api-access-vq52l\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.042574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.042487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6jwl\" (UniqueName: \"kubernetes.io/projected/5f92c179-25c7-4c7a-89fb-286028524d9f-kube-api-access-h6jwl\") pod \"service-ca-operator-859f94c4b-5zfcb\" (UID: \"5f92c179-25c7-4c7a-89fb-286028524d9f\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.042878 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.042618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/66e4f7f3-6970-4189-b923-865fbac766d2-serving-cert\") pod \"insights-operator-76bcd6ccf5-rp5wd\" (UID: \"66e4f7f3-6970-4189-b923-865fbac766d2\") " pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.044295 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.044270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.044375 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.044340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k5prd\" (UniqueName: \"kubernetes.io/projected/5613ea48-561a-49c6-84da-94fa313b1668-kube-api-access-k5prd\") pod \"kube-storage-version-migrator-operator-8d7dd4675-5gsr9\" (UID: \"5613ea48-561a-49c6-84da-94fa313b1668\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.044441 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.044419    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m2fb7\" (UniqueName: \"kubernetes.io/projected/22caec24-941d-43d1-b74a-e513d530c79e-kube-api-access-m2fb7\") pod \"network-check-source-685456655f-nmtv2\" (UID: \"22caec24-941d-43d1-b74a-e513d530c79e\") " pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2"
Sep 09 12:48:57.044601 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.044539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qt8h6\" (UniqueName: \"kubernetes.io/projected/45fa2142-a4b5-405b-80a4-27438f6e9080-kube-api-access-qt8h6\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.046302 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.046272    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tknt4\" (UniqueName: \"kubernetes.io/projected/42bd9e84-8866-4e8c-a958-a96cd635510b-kube-api-access-tknt4\") pod \"console-operator-85b454874b-cg7vj\" (UID: \"42bd9e84-8866-4e8c-a958-a96cd635510b\") " pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.047602 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.047580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfmrp\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.048022 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.048005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbmsb\" (UniqueName: \"kubernetes.io/projected/6153e58f-438f-44f4-a984-421cae427aea-kube-api-access-tbmsb\") pod \"ingress-canary-2l9tv\" (UID: \"6153e58f-438f-44f4-a984-421cae427aea\") " pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:57.048462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.048431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb4jr\" (UniqueName: \"kubernetes.io/projected/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-kube-api-access-pb4jr\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:57.067714 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.067564    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"
Sep 09 12:48:57.093707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.093555    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"
Sep 09 12:48:57.120269 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.120117    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:48:57.127330 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.127305    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"
Sep 09 12:48:57.141672 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.141156    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd"
Sep 09 12:48:57.148959 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.148590    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2"
Sep 09 12:48:57.156000 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.155679    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2l9tv"
Sep 09 12:48:57.360399 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.360368    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54"]
Sep 09 12:48:57.364593 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:57.364512    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6607ac44_3eff_4dbc_982f_8c3902f790ee.slice/crio-85aff5f4cac0607e1a310471e4517ad30fc34d233c85ab1b9fc5f7ee83bbaf59 WatchSource:0}: Error finding container 85aff5f4cac0607e1a310471e4517ad30fc34d233c85ab1b9fc5f7ee83bbaf59: Status 404 returned error can't find the container with id 85aff5f4cac0607e1a310471e4517ad30fc34d233c85ab1b9fc5f7ee83bbaf59
Sep 09 12:48:57.382687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.382660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb"]
Sep 09 12:48:57.384659 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:57.384636    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f92c179_25c7_4c7a_89fb_286028524d9f.slice/crio-17bc1bfa5a72b5053721065c8a17c098c667c2dc6e5cf0e64bbf8b264a9d2478 WatchSource:0}: Error finding container 17bc1bfa5a72b5053721065c8a17c098c667c2dc6e5cf0e64bbf8b264a9d2478: Status 404 returned error can't find the container with id 17bc1bfa5a72b5053721065c8a17c098c667c2dc6e5cf0e64bbf8b264a9d2478
Sep 09 12:48:57.431267 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.431208    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-cg7vj"]
Sep 09 12:48:57.433899 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:57.433873    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42bd9e84_8866_4e8c_a958_a96cd635510b.slice/crio-cedfe48b1ef568ae7dfee216268ecec178b3a31ae121686e754c5f8f5ed0c7c7 WatchSource:0}: Error finding container cedfe48b1ef568ae7dfee216268ecec178b3a31ae121686e754c5f8f5ed0c7c7: Status 404 returned error can't find the container with id cedfe48b1ef568ae7dfee216268ecec178b3a31ae121686e754c5f8f5ed0c7c7
Sep 09 12:48:57.445206 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.445186    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9"]
Sep 09 12:48:57.446897 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.446705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:48:57.446897 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.446721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:48:57.447031 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.446721    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:48:57.449805 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.449269    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ns67g\""
Sep 09 12:48:57.449805 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.449707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:48:57.450013 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.449844    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:48:57.450095 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.450043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcx5j\""
Sep 09 12:48:57.450805 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:57.450685    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5613ea48_561a_49c6_84da_94fa313b1668.slice/crio-eee4210f8543771b0699f5c4563e3cc9582d646f974120d0c2ebfba904326316 WatchSource:0}: Error finding container eee4210f8543771b0699f5c4563e3cc9582d646f974120d0c2ebfba904326316: Status 404 returned error can't find the container with id eee4210f8543771b0699f5c4563e3cc9582d646f974120d0c2ebfba904326316
Sep 09 12:48:57.469161 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.469134    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-nmtv2"]
Sep 09 12:48:57.471401 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.471378    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2l9tv"]
Sep 09 12:48:57.472729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.472710    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-rp5wd"]
Sep 09 12:48:57.475932 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:48:57.475912    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66e4f7f3_6970_4189_b923_865fbac766d2.slice/crio-0530f2643143d5447e7b5614be0c287bf26ce35e68b85908c9a3bc1d5794327b WatchSource:0}: Error finding container 0530f2643143d5447e7b5614be0c287bf26ce35e68b85908c9a3bc1d5794327b: Status 404 returned error can't find the container with id 0530f2643143d5447e7b5614be0c287bf26ce35e68b85908c9a3bc1d5794327b
Sep 09 12:48:57.535673 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:57.535799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535682    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:57.535799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535713    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:57.535799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535736    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.535799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535765    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:57.535799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.535796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535813    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535875    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535817    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535908    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535875    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.535855527 +0000 UTC m=+36.571103432 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535909    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535821    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.535986    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.535957705 +0000 UTC m=+36.571205603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536007    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.535996841 +0000 UTC m=+36.571244737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536021    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.536012452 +0000 UTC m=+36.571260337 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536036    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.536028058 +0000 UTC m=+36.571275948 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:48:57.536055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.536059    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:57.536646 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536116    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.536106205 +0000 UTC m=+36.571354124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:48:57.536646 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536129    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:57.536646 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:57.536170    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:48:58.536159493 +0000 UTC m=+36.571407401 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:57.594806 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.594778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2l9tv" event={"ID":"6153e58f-438f-44f4-a984-421cae427aea","Type":"ContainerStarted","Data":"2da2b15ba095aaa0d62bb64d061c6d743d1f3e3572953e88f75fb5207c7886ab"}
Sep 09 12:48:57.596038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.596013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54" event={"ID":"6607ac44-3eff-4dbc-982f-8c3902f790ee","Type":"ContainerStarted","Data":"85aff5f4cac0607e1a310471e4517ad30fc34d233c85ab1b9fc5f7ee83bbaf59"}
Sep 09 12:48:57.597216 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.597196    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2" event={"ID":"22caec24-941d-43d1-b74a-e513d530c79e","Type":"ContainerStarted","Data":"603791d0385ed4bce87646a3b2604f9b0469354c5036d9e5da0bb3ddeca20d76"}
Sep 09 12:48:57.598660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.598637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd" event={"ID":"66e4f7f3-6970-4189-b923-865fbac766d2","Type":"ContainerStarted","Data":"0530f2643143d5447e7b5614be0c287bf26ce35e68b85908c9a3bc1d5794327b"}
Sep 09 12:48:57.599887 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.599865    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb" event={"ID":"5f92c179-25c7-4c7a-89fb-286028524d9f","Type":"ContainerStarted","Data":"17bc1bfa5a72b5053721065c8a17c098c667c2dc6e5cf0e64bbf8b264a9d2478"}
Sep 09 12:48:57.601046 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.601028    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9" event={"ID":"5613ea48-561a-49c6-84da-94fa313b1668","Type":"ContainerStarted","Data":"eee4210f8543771b0699f5c4563e3cc9582d646f974120d0c2ebfba904326316"}
Sep 09 12:48:57.602295 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.602273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" event={"ID":"42bd9e84-8866-4e8c-a958-a96cd635510b","Type":"ContainerStarted","Data":"cedfe48b1ef568ae7dfee216268ecec178b3a31ae121686e754c5f8f5ed0c7c7"}
Sep 09 12:48:57.795651 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.795573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:57.896328 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:57.896296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:58.544545 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544487    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544581    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544707    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544734    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.544771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544680    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544870    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544876    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.544856542 +0000 UTC m=+38.580104443 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544891    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544756    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544775    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544929    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544797    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544901    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.544887352 +0000 UTC m=+38.580135243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.544992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.544981989 +0000 UTC m=+38.580229877 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:48:58.545257 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.545005    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.544996833 +0000 UTC m=+38.580244722 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:48:58.546058 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.545026    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.545017723 +0000 UTC m=+38.580265614 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:48:58.546058 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.545045    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.545036943 +0000 UTC m=+38.580284834 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:48:58.546058 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:48:58.545062    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:00.545055267 +0000 UTC m=+38.580303152 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:48:58.811268 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.811183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:58.911984 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:58.911946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:59.013617 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:59.013583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:59.837923 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:59.837764    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:48:59.935314 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:48:59.935263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565383    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565446    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565431024 +0000 UTC m=+42.600678926 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565472    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565540    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565523848 +0000 UTC m=+42.600771738 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565598    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565622    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565614574 +0000 UTC m=+42.600862459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565663    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565657207 +0000 UTC m=+42.600905091 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565699    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565726    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565720576 +0000 UTC m=+42.600968461 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:49:00.565911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:00.565795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565863    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565870    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565886    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565890    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565883895 +0000 UTC m=+42.601131780 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:49:00.566876 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:00.565929    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:04.565919063 +0000 UTC m=+42.601166949 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:01.010013 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:01.009977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:02.776475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:02.776441    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:04.071733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.071693    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:04.605643 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605617    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:04.605767 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605656    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:04.605767 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:04.605767 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605731    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:04.605767 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:04.605767 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605761    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605773    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605821    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605804287 +0000 UTC m=+50.641052178 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605824    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605837    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605830355 +0000 UTC m=+50.641078239 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605867    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.605876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605906    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605889671 +0000 UTC m=+50.641137572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605870    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605925    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:49:04.605951 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605953    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605940485 +0000 UTC m=+50.641188373 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:49:04.606255 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605975    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605969154 +0000 UTC m=+50.641217039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:49:04.606255 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.605986    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.605981456 +0000 UTC m=+50.641229340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:49:04.606255 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:04.606013    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:04.606255 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.606084    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:49:04.606255 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:04.606111    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:12.60610524 +0000 UTC m=+50.641353124 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:49:05.544313 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:05.544246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:05.644622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:05.644583    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:06.629584 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.629556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/0.log"
Sep 09 12:49:06.630129 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.629603    2580 generic.go:356] "Generic (PLEG): container finished" podID="42bd9e84-8866-4e8c-a958-a96cd635510b" containerID="99730483850e34bb02785beb7a4d7e49b910922b6580a31fa4f988ee2a28e323" exitCode=255
Sep 09 12:49:06.630129 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.629639    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" event={"ID":"42bd9e84-8866-4e8c-a958-a96cd635510b","Type":"ContainerDied","Data":"99730483850e34bb02785beb7a4d7e49b910922b6580a31fa4f988ee2a28e323"}
Sep 09 12:49:06.630129 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.629942    2580 scope.go:117] "RemoveContainer" containerID="99730483850e34bb02785beb7a4d7e49b910922b6580a31fa4f988ee2a28e323"
Sep 09 12:49:06.631716 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.631679    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2l9tv" event={"ID":"6153e58f-438f-44f4-a984-421cae427aea","Type":"ContainerStarted","Data":"ff3b6a34b283362ca1cc9a9e58f8d6483abd1f60f671551e65ac4bb4a7e5c75b"}
Sep 09 12:49:06.633103 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.633079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54" event={"ID":"6607ac44-3eff-4dbc-982f-8c3902f790ee","Type":"ContainerStarted","Data":"1e1dda6e40f957d7e8ec68fdf7c4a073d8c3ec51f8d447525a1998dcc054e233"}
Sep 09 12:49:06.635886 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.635859    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2" event={"ID":"22caec24-941d-43d1-b74a-e513d530c79e","Type":"ContainerStarted","Data":"e0b3ead74127dc9fa435fdc59d04e32a4565a73725c884929d3bbd5e3a1a0e90"}
Sep 09 12:49:06.637544 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.637518    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd" event={"ID":"66e4f7f3-6970-4189-b923-865fbac766d2","Type":"ContainerStarted","Data":"98e9bf69a1fbdca1ec37d3714898a68fa83deec9cc576526bc5ff6d3bccb198a"}
Sep 09 12:49:06.639620 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.639599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb" event={"ID":"5f92c179-25c7-4c7a-89fb-286028524d9f","Type":"ContainerStarted","Data":"6681605d8a1cd07b357d8847aa67e5b6318409b6857cd4e6dc49ace3921365ad"}
Sep 09 12:49:06.642948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.642923    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="cd808026cad0af47e26a8390dbb1d96c5eaf3e6577fa8b4f741166dde862abf1" exitCode=0
Sep 09 12:49:06.643049 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.642993    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"cd808026cad0af47e26a8390dbb1d96c5eaf3e6577fa8b4f741166dde862abf1"}
Sep 09 12:49:06.644861 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.644839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9" event={"ID":"5613ea48-561a-49c6-84da-94fa313b1668","Type":"ContainerStarted","Data":"a63e88bd06ddceedeac58e0dd7b4fbb7a19c26d9638e0ec17c4a987f687a6dde"}
Sep 09 12:49:06.662890 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.662851    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-nmtv2" podStartSLOduration=10.029237348 podStartE2EDuration="18.662814142s" totalImagesPullingTime="8.633576794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.662789075 +0000 UTC m=+44.698036982" watchObservedRunningTime="2026-09-09 12:49:06.662814142 +0000 UTC m=+44.698062028"
Sep 09 12:49:06.705571 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.705058    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-695c895bd6-z9s54" podStartSLOduration=11.253415558 podStartE2EDuration="19.705038806s" totalImagesPullingTime="8.451623248s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.7030096 +0000 UTC m=+44.738257508" watchObservedRunningTime="2026-09-09 12:49:06.705038806 +0000 UTC m=+44.740286715"
Sep 09 12:49:06.769561 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.769108    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9" podStartSLOduration=11.366261911 podStartE2EDuration="19.769088784s" totalImagesPullingTime="8.402826873s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.767763034 +0000 UTC m=+44.803010943" watchObservedRunningTime="2026-09-09 12:49:06.769088784 +0000 UTC m=+44.804336692"
Sep 09 12:49:06.770781 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.770740    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb" podStartSLOduration=11.308210805 podStartE2EDuration="19.770726844s" totalImagesPullingTime="8.462516039s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.74590747 +0000 UTC m=+44.781155378" watchObservedRunningTime="2026-09-09 12:49:06.770726844 +0000 UTC m=+44.805974753"
Sep 09 12:49:06.812517 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.811191    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd" podStartSLOduration=11.542002697000001 podStartE2EDuration="19.811172002s" totalImagesPullingTime="8.269169305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.790781209 +0000 UTC m=+44.826029119" watchObservedRunningTime="2026-09-09 12:49:06.811172002 +0000 UTC m=+44.846419902"
Sep 09 12:49:06.812517 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.811303    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2l9tv" podStartSLOduration=2.43741654 podStartE2EDuration="10.811297525s" totalImagesPullingTime="8.373880985s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:06.810199898 +0000 UTC m=+44.845447812" watchObservedRunningTime="2026-09-09 12:49:06.811297525 +0000 UTC m=+44.846545433"
Sep 09 12:49:06.812517 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.811933    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:06.911909 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:06.911858    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:07.121316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.121275    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:07.121463 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.121329    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:07.208495 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.208427    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"]
Sep 09 12:49:07.218057 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.218026    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:07.239468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.239447    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"]
Sep 09 12:49:07.239622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.239556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"
Sep 09 12:49:07.241841 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.241815    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qv2wf\""
Sep 09 12:49:07.241957 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.241817    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 12:49:07.242074 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.242052    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 12:49:07.303735 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.303706    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-hxhf7"]
Sep 09 12:49:07.307911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.307880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:07.327401 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.327373    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hxhf7"]
Sep 09 12:49:07.327572 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.327497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.329698 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.329675    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s29r7\""
Sep 09 12:49:07.329816 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.329797    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 12:49:07.330178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.330160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 12:49:07.333269 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.333248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5d8v\" (UniqueName: \"kubernetes.io/projected/b65e71dc-6b03-438b-aa9d-20dd02df2f1b-kube-api-access-f5d8v\") pod \"migrator-fdbb8fdff-sp942\" (UID: \"b65e71dc-6b03-438b-aa9d-20dd02df2f1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"
Sep 09 12:49:07.367776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.367742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:07.433876 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.433836    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82213e13-c14f-4841-9ae5-7fbbe5f150be-data-volume\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.434055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.433898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82213e13-c14f-4841-9ae5-7fbbe5f150be-crio-socket\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.434055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.433948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5d8v\" (UniqueName: \"kubernetes.io/projected/b65e71dc-6b03-438b-aa9d-20dd02df2f1b-kube-api-access-f5d8v\") pod \"migrator-fdbb8fdff-sp942\" (UID: \"b65e71dc-6b03-438b-aa9d-20dd02df2f1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"
Sep 09 12:49:07.434055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.433996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.434055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.434033    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.434282 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.434113    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8xv8\" (UniqueName: \"kubernetes.io/projected/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-api-access-f8xv8\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.442290 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.442266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5d8v\" (UniqueName: \"kubernetes.io/projected/b65e71dc-6b03-438b-aa9d-20dd02df2f1b-kube-api-access-f5d8v\") pod \"migrator-fdbb8fdff-sp942\" (UID: \"b65e71dc-6b03-438b-aa9d-20dd02df2f1b\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"
Sep 09 12:49:07.535232 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82213e13-c14f-4841-9ae5-7fbbe5f150be-crio-socket\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535232 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535225    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535373 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:07.535285    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:07.535373 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:07.535327    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:08.035314942 +0000 UTC m=+46.070562827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:07.535373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/82213e13-c14f-4841-9ae5-7fbbe5f150be-crio-socket\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8xv8\" (UniqueName: \"kubernetes.io/projected/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-api-access-f8xv8\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535576 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:07.535517    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:07.535576 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82213e13-c14f-4841-9ae5-7fbbe5f150be-data-volume\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.535674 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:07.535583    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:08.035567585 +0000 UTC m=+46.070815474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:07.535854 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.535831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/82213e13-c14f-4841-9ae5-7fbbe5f150be-data-volume\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.543132 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.543114    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8xv8\" (UniqueName: \"kubernetes.io/projected/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-api-access-f8xv8\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:07.550937 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.550920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"
Sep 09 12:49:07.652075 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.652042    2580 generic.go:356] "Generic (PLEG): container finished" podID="421a4317-7258-4c56-a856-4a2e63df7522" containerID="66a73f791ea1378292bfd47664f8a5509b2c4789729b4b9d34e82ccde37e1665" exitCode=0
Sep 09 12:49:07.652466 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.652121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerDied","Data":"66a73f791ea1378292bfd47664f8a5509b2c4789729b4b9d34e82ccde37e1665"}
Sep 09 12:49:07.653725 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.653707    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/1.log"
Sep 09 12:49:07.654097 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.654084    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/0.log"
Sep 09 12:49:07.654160 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.654113    2580 generic.go:356] "Generic (PLEG): container finished" podID="42bd9e84-8866-4e8c-a958-a96cd635510b" containerID="a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a" exitCode=255
Sep 09 12:49:07.654227 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.654208    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" event={"ID":"42bd9e84-8866-4e8c-a958-a96cd635510b","Type":"ContainerDied","Data":"a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a"}
Sep 09 12:49:07.654265 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.654250    2580 scope.go:117] "RemoveContainer" containerID="99730483850e34bb02785beb7a4d7e49b910922b6580a31fa4f988ee2a28e323"
Sep 09 12:49:07.654373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.654356    2580 scope.go:117] "RemoveContainer" containerID="a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a"
Sep 09 12:49:07.654571 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:07.654545    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cg7vj_openshift-console-operator(42bd9e84-8866-4e8c-a958-a96cd635510b)\"" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" podUID="42bd9e84-8866-4e8c-a958-a96cd635510b"
Sep 09 12:49:07.826853 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:07.826823    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942"]
Sep 09 12:49:07.829258 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:07.829234    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb65e71dc_6b03_438b_aa9d_20dd02df2f1b.slice/crio-1d2a10828e0b971bb103437ffea5dcaa5d114b060022049c003ef7dda6ad7c42 WatchSource:0}: Error finding container 1d2a10828e0b971bb103437ffea5dcaa5d114b060022049c003ef7dda6ad7c42: Status 404 returned error can't find the container with id 1d2a10828e0b971bb103437ffea5dcaa5d114b060022049c003ef7dda6ad7c42
Sep 09 12:49:08.041279 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.041197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:08.041279 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.041236    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:08.041481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:08.041331    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:08.041481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:08.041345    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:08.041481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:08.041389    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.041377033 +0000 UTC m=+47.076624919 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:08.041481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:08.041401    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:09.041395757 +0000 UTC m=+47.076643641 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:08.420190 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.420155    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:08.525517 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.525468    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:08.661149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.661116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" event={"ID":"421a4317-7258-4c56-a856-4a2e63df7522","Type":"ContainerStarted","Data":"5335f7a41c92053884327f3efd2fa69c7708afbf118b5688df048737cc996ab7"}
Sep 09 12:49:08.662539 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.662522    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/1.log"
Sep 09 12:49:08.662847 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.662824    2580 scope.go:117] "RemoveContainer" containerID="a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a"
Sep 09 12:49:08.662998 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:08.662974    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-cg7vj_openshift-console-operator(42bd9e84-8866-4e8c-a958-a96cd635510b)\"" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" podUID="42bd9e84-8866-4e8c-a958-a96cd635510b"
Sep 09 12:49:08.663758 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.663736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942" event={"ID":"b65e71dc-6b03-438b-aa9d-20dd02df2f1b","Type":"ContainerStarted","Data":"1d2a10828e0b971bb103437ffea5dcaa5d114b060022049c003ef7dda6ad7c42"}
Sep 09 12:49:08.685110 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.684888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-rjk6k" podStartSLOduration=10.773146093 podStartE2EDuration="45.684870653s" totalImagesPullingTime="34.91172456s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:08.683221006 +0000 UTC m=+46.718468915" watchObservedRunningTime="2026-09-09 12:49:08.684870653 +0000 UTC m=+46.720118563"
Sep 09 12:49:08.805887 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.805855    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:08.906201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.906167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:08.959017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:08.958941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:09.051882 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.051848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:09.052061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.051898    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:09.052061 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:09.051993    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:09.052161 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:09.052073    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:09.052161 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:09.052090    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:11.052070534 +0000 UTC m=+49.087318423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:09.052161 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:09.052143    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:11.052127087 +0000 UTC m=+49.087374987 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:09.079764 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.079740    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8ngjq_853f7946-ed3b-44b7-a2d8-fc6e57cf18b0/dns-node-resolver/0.log"
Sep 09 12:49:09.170172 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.170142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:09.270527 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.270428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:09.569653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.569579    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:09.669311 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:09.669286    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:10.279969 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.279942    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rc72g_29c984e6-6c2f-4784-bb63-3b13e74b7af6/node-ca/0.log"
Sep 09 12:49:10.671661 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.671627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942" event={"ID":"b65e71dc-6b03-438b-aa9d-20dd02df2f1b","Type":"ContainerStarted","Data":"6e876d0ac6366f33cd77440cffd3d103bfc3a1d61721d5f683957d4b9c8b1d4c"}
Sep 09 12:49:10.672005 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.671674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942" event={"ID":"b65e71dc-6b03-438b-aa9d-20dd02df2f1b","Type":"ContainerStarted","Data":"7f9615c4e2fa92ce970fd6a61135aaa3ef5515f9936a4e4ddccb71c275a2c174"}
Sep 09 12:49:10.686519 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.686463    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-sp942" podStartSLOduration=1.902174571 podStartE2EDuration="3.68644903s" totalImagesPullingTime="1.784274459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:10.685758103 +0000 UTC m=+48.721006010" watchObservedRunningTime="2026-09-09 12:49:10.68644903 +0000 UTC m=+48.721696937"
Sep 09 12:49:10.778059 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.778031    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:10.830893 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.830836    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:10.879867 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:10.879844    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l9tv_6153e58f-438f-44f4-a984-421cae427aea/serve-healthcheck-canary/0.log"
Sep 09 12:49:11.069104 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.069046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:11.069104 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.069082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:11.069247 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:11.069172    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 12:49:11.069247 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:11.069227    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:15.069214196 +0000 UTC m=+53.104462081 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : configmap "kube-rbac-proxy" not found
Sep 09 12:49:11.069320 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:11.069259    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:11.069320 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:11.069309    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:15.069296607 +0000 UTC m=+53.104544495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:11.081493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.081449    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sp942_b65e71dc-6b03-438b-aa9d-20dd02df2f1b/migrator/0.log"
Sep 09 12:49:11.282176 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.282151    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sp942_b65e71dc-6b03-438b-aa9d-20dd02df2f1b/graceful-termination/0.log"
Sep 09 12:49:11.482299 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.482274    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5gsr9_5613ea48-561a-49c6-84da-94fa313b1668/kube-storage-version-migrator-operator/0.log"
Sep 09 12:49:11.650279 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:11.650253    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:12.681956 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.681920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.681974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.682005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.682035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.682052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682060    2580 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.682070    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:12.682091    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682138    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls podName:7f392a16-e3e2-4bb1-b745-2575ec7debe7 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682117799 +0000 UTC m=+66.717365685 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6cdcc96965-9kn6t" (UID: "7f392a16-e3e2-4bb1-b745-2575ec7debe7") : secret "cluster-monitoring-operator-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682155    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682174    2580 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682186    2580 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682204    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls podName:45fa2142-a4b5-405b-80a4-27438f6e9080 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682192517 +0000 UTC m=+66.717440415 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls") pod "dns-default-zr2l7" (UID: "45fa2142-a4b5-405b-80a4-27438f6e9080") : secret "dns-default-metrics-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682220    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert podName:cad0a6c1-e143-45f1-a7f3-08826bc0c048 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682212191 +0000 UTC m=+66.717460089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-fj4kt" (UID: "cad0a6c1-e143-45f1-a7f3-08826bc0c048") : secret "networking-console-plugin-cert" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682230    2580 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682194    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682245    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6c88ccbb9b-zjlhw: secret "image-registry-tls" not found
Sep 09 12:49:12.682387 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682234    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682226724 +0000 UTC m=+66.717474614 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : configmap references non-existent config key: service-ca.crt
Sep 09 12:49:12.682961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682285    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs podName:e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682268095 +0000 UTC m=+66.717515983 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs") pod "router-default-b548b666-qdt2z" (UID: "e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3") : secret "router-metrics-certs-default" not found
Sep 09 12:49:12.682961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682298    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls podName:a163c869-c9d7-42cf-a8cc-8c7aaf2594b6 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.682292008 +0000 UTC m=+66.717539896 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls") pod "cluster-samples-operator-7cdcfcf76c-84nxz" (UID: "a163c869-c9d7-42cf-a8cc-8c7aaf2594b6") : secret "samples-operator-tls" not found
Sep 09 12:49:12.682961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:12.682307    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls podName:dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:28.68230212 +0000 UTC m=+66.717550004 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls") pod "image-registry-6c88ccbb9b-zjlhw" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608") : secret "image-registry-tls" not found
Sep 09 12:49:13.213240 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:13.213207    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:13.679573 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:13.679540    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:13.819224 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:13.819196    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:14.567686 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:14.567652    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:15.102687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:15.102654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:15.102687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:15.102693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:15.103222 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:15.102860    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:15.103222 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:15.102914    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls podName:82213e13-c14f-4841-9ae5-7fbbe5f150be nodeName:}" failed. No retries permitted until 2026-09-09 12:49:23.102901136 +0000 UTC m=+61.138149021 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls") pod "insights-runtime-extractor-hxhf7" (UID: "82213e13-c14f-4841-9ae5-7fbbe5f150be") : secret "insights-runtime-extractor-tls" not found
Sep 09 12:49:15.103382 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:15.103358    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/82213e13-c14f-4841-9ae5-7fbbe5f150be-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:15.779581 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:15.779539    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:15.833900 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:15.833867    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:16.215966 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:16.215935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:16.470052 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:16.469957    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:16.684400 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:16.684369    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:17.120440 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.120409    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:17.120621 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.120485    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:17.120827 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.120815    2580 scope.go:117] "RemoveContainer" containerID="a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a"
Sep 09 12:49:17.277231 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.277202    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:17.372656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.372581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:17.699513 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.699475    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:49:17.699906 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.699892    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/1.log"
Sep 09 12:49:17.700002 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.699922    2580 generic.go:356] "Generic (PLEG): container finished" podID="42bd9e84-8866-4e8c-a958-a96cd635510b" containerID="34f589df0c89427c85d6e0d1a678200a22acea6ebfcb9077eb567c08fba649a6" exitCode=255
Sep 09 12:49:17.700002 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.699959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" event={"ID":"42bd9e84-8866-4e8c-a958-a96cd635510b","Type":"ContainerDied","Data":"34f589df0c89427c85d6e0d1a678200a22acea6ebfcb9077eb567c08fba649a6"}
Sep 09 12:49:17.700068 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.700002    2580 scope.go:117] "RemoveContainer" containerID="a41a8a707fd472c4eb25ab18761ee43c3465e744c7f0f7f10c1de434db43e08a"
Sep 09 12:49:17.700287 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:17.700269    2580 scope.go:117] "RemoveContainer" containerID="34f589df0c89427c85d6e0d1a678200a22acea6ebfcb9077eb567c08fba649a6"
Sep 09 12:49:17.700460 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:17.700440    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-cg7vj_openshift-console-operator(42bd9e84-8866-4e8c-a958-a96cd635510b)\"" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" podUID="42bd9e84-8866-4e8c-a958-a96cd635510b"
Sep 09 12:49:18.098864 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.098781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:18.298399 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.298363    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:18.500615 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.500582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:18.595624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.595599    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:18.704433 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.704411    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:49:18.708997 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.708968    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:18.895871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:18.895844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:19.636220 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:19.636180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:19.899340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:19.899305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:20.093417 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:20.093385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:20.294053 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:20.293970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:20.878021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:20.877955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:20.928739 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:20.928708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:21.894038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:21.894006    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:22.294185 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.294077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:22.395454 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.395414    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:22.506063 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.506023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:22.567175 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.567105    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:22.591531 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.591498    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-qdwjh"
Sep 09 12:49:22.702348 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:22.702323    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:23.040165 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.040140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:23.175989 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.175964    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:23.178281 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.178255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/82213e13-c14f-4841-9ae5-7fbbe5f150be-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-hxhf7\" (UID: \"82213e13-c14f-4841-9ae5-7fbbe5f150be\") " pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:23.239263 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.239239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-s29r7\""
Sep 09 12:49:23.248047 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.248031    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-hxhf7"
Sep 09 12:49:23.515077 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.515038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-hxhf7"]
Sep 09 12:49:23.516988 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:23.516962    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82213e13_c14f_4841_9ae5_7fbbe5f150be.slice/crio-f449fb68fa36bb3204d14e4d8b18089fc2b6df1626709589e545bb9d2a51af50 WatchSource:0}: Error finding container f449fb68fa36bb3204d14e4d8b18089fc2b6df1626709589e545bb9d2a51af50: Status 404 returned error can't find the container with id f449fb68fa36bb3204d14e4d8b18089fc2b6df1626709589e545bb9d2a51af50
Sep 09 12:49:23.720226 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.720192    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hxhf7" event={"ID":"82213e13-c14f-4841-9ae5-7fbbe5f150be","Type":"ContainerStarted","Data":"89fbfa7575040980770b1c9fba51473e6dc88c6e2d61c7248f9ed9250f611ba8"}
Sep 09 12:49:23.720391 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:23.720254    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hxhf7" event={"ID":"82213e13-c14f-4841-9ae5-7fbbe5f150be","Type":"ContainerStarted","Data":"f449fb68fa36bb3204d14e4d8b18089fc2b6df1626709589e545bb9d2a51af50"}
Sep 09 12:49:24.097541 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:24.097494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:24.296625 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:24.296595    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:24.506450 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:24.506417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:24.697755 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:24.697719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:24.728553 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:24.728477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hxhf7" event={"ID":"82213e13-c14f-4841-9ae5-7fbbe5f150be","Type":"ContainerStarted","Data":"4827621b341252b61606eb8febb4ae5924403c1e536d6ffe54fa6dafdf506b9d"}
Sep 09 12:49:25.662947 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:25.662909    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.736356 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:26.736316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-hxhf7" event={"ID":"82213e13-c14f-4841-9ae5-7fbbe5f150be","Type":"ContainerStarted","Data":"135a14eabcb259546bbc812b6f7830690f5ab985071122eb5945877e7293f10a"}
Sep 09 12:49:26.753619 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:26.753569    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-hxhf7" podStartSLOduration=17.507766899 podStartE2EDuration="19.753554068s" totalImagesPullingTime="2.245787169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:49:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:26.751840243 +0000 UTC m=+64.787088150" watchObservedRunningTime="2026-09-09 12:49:26.753554068 +0000 UTC m=+64.788801974"
Sep 09 12:49:26.814651 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:26.814618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:26.915228 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:26.915204    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:27.015589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:27.015521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:27.120493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:27.120455    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:27.120652 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:27.120530    2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:27.120881 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:27.120863    2580 scope.go:117] "RemoveContainer" containerID="34f589df0c89427c85d6e0d1a678200a22acea6ebfcb9077eb567c08fba649a6"
Sep 09 12:49:27.121046 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:27.121027    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-85b454874b-cg7vj_openshift-console-operator(42bd9e84-8866-4e8c-a958-a96cd635510b)\"" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" podUID="42bd9e84-8866-4e8c-a958-a96cd635510b"
Sep 09 12:49:28.220054 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.220003    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:49:28.226309 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.226283    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 12:49:28.233211 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.233194    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799-metrics-certs\") pod \"network-metrics-daemon-q7pfr\" (UID: \"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799\") " pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:49:28.320829 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.320803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:49:28.323554 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.323537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 12:49:28.333414 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.333390    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/abd06375-8c1e-49ec-9a04-9782206cadbc-original-pull-secret\") pod \"global-pull-secret-syncer-fj9l6\" (UID: \"abd06375-8c1e-49ec-9a04-9782206cadbc\") " pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:49:28.369871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.369842    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-fj9l6"
Sep 09 12:49:28.383066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.383047    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ns67g\""
Sep 09 12:49:28.391798 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.391779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-q7pfr"
Sep 09 12:49:28.421983 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.421958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:49:28.427380 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.425877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq4hn\" (UniqueName: \"kubernetes.io/projected/43acb564-ce77-4108-a5be-23fe3d0ea28e-kube-api-access-bq4hn\") pod \"network-check-target-hkb9w\" (UID: \"43acb564-ce77-4108-a5be-23fe3d0ea28e\") " pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:49:28.639265 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.639197    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-fj9l6"]
Sep 09 12:49:28.641857 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:28.641833    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabd06375_8c1e_49ec_9a04_9782206cadbc.slice/crio-fd99a8b39c5cf7826ebefc896012b9ae941881122a1f1fc723382bb7b27fa017 WatchSource:0}: Error finding container fd99a8b39c5cf7826ebefc896012b9ae941881122a1f1fc723382bb7b27fa017: Status 404 returned error can't find the container with id fd99a8b39c5cf7826ebefc896012b9ae941881122a1f1fc723382bb7b27fa017
Sep 09 12:49:28.663754 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.663735    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qcx5j\""
Sep 09 12:49:28.666630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.666607    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-q7pfr"]
Sep 09 12:49:28.668896 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:28.668875    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode26aeb9a_4ba3_42ce_9b66_7d8cf6fc3799.slice/crio-b40ce06bedb2e1dd1c60685affd8ae900146f3be9782b6559c701d829f803f15 WatchSource:0}: Error finding container b40ce06bedb2e1dd1c60685affd8ae900146f3be9782b6559c701d829f803f15: Status 404 returned error can't find the container with id b40ce06bedb2e1dd1c60685affd8ae900146f3be9782b6559c701d829f803f15
Sep 09 12:49:28.671952 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.671936    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:49:28.725300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725110    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:28.725300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:28.725300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:28.725300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725283    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:28.725619 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725321    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:28.725619 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:28.725619 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.725372    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:28.726119 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.726095    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-service-ca-bundle\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:28.728068 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.728042    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/cad0a6c1-e143-45f1-a7f3-08826bc0c048-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-fj4kt\" (UID: \"cad0a6c1-e143-45f1-a7f3-08826bc0c048\") " pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:28.728519 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.728486    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3-metrics-certs\") pod \"router-default-b548b666-qdt2z\" (UID: \"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3\") " pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:28.728607 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.728562    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7f392a16-e3e2-4bb1-b745-2575ec7debe7-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6cdcc96965-9kn6t\" (UID: \"7f392a16-e3e2-4bb1-b745-2575ec7debe7\") " pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:28.728607 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.728593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/a163c869-c9d7-42cf-a8cc-8c7aaf2594b6-samples-operator-tls\") pod \"cluster-samples-operator-7cdcfcf76c-84nxz\" (UID: \"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:28.728905 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.728877    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/45fa2142-a4b5-405b-80a4-27438f6e9080-metrics-tls\") pod \"dns-default-zr2l7\" (UID: \"45fa2142-a4b5-405b-80a4-27438f6e9080\") " pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:28.729027 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.729005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"image-registry-6c88ccbb9b-zjlhw\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") " pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:28.743871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.743839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7pfr" event={"ID":"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799","Type":"ContainerStarted","Data":"b40ce06bedb2e1dd1c60685affd8ae900146f3be9782b6559c701d829f803f15"}
Sep 09 12:49:28.744925 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.744902    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fj9l6" event={"ID":"abd06375-8c1e-49ec-9a04-9782206cadbc","Type":"ContainerStarted","Data":"fd99a8b39c5cf7826ebefc896012b9ae941881122a1f1fc723382bb7b27fa017"}
Sep 09 12:49:28.880940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.880917    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-qkhdl\""
Sep 09 12:49:28.889704 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.889646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:28.907738 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.907715    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-h4fmd\""
Sep 09 12:49:28.915807 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.915788    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-j2chm\""
Sep 09 12:49:28.915907 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.915818    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"
Sep 09 12:49:28.923995 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.923972    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:28.936730 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.936541    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-g6xt9\""
Sep 09 12:49:28.936958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.936939    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-hkb9w"]
Sep 09 12:49:28.942398 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:28.942354    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43acb564_ce77_4108_a5be_23fe3d0ea28e.slice/crio-26b1b83dcceea8b8e5dfe3460fe4772340301292fd2a575372bfea3fa166917a WatchSource:0}: Error finding container 26b1b83dcceea8b8e5dfe3460fe4772340301292fd2a575372bfea3fa166917a: Status 404 returned error can't find the container with id 26b1b83dcceea8b8e5dfe3460fe4772340301292fd2a575372bfea3fa166917a
Sep 09 12:49:28.945000 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.944772    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"
Sep 09 12:49:28.964114 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.964078    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-sqpbh\""
Sep 09 12:49:28.968522 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.968322    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-pmjpj\""
Sep 09 12:49:28.974285 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.972070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt"
Sep 09 12:49:28.974622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:28.974570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:29.166218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.166192    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:49:29.168638 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:29.168617    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfdef752_22f2_4bf6_a3a0_fc9f9bfb3608.slice/crio-5a313b2c17b271464da8567224374592607abfdda0fc7cb2ca88007bfa17e543 WatchSource:0}: Error finding container 5a313b2c17b271464da8567224374592607abfdda0fc7cb2ca88007bfa17e543: Status 404 returned error can't find the container with id 5a313b2c17b271464da8567224374592607abfdda0fc7cb2ca88007bfa17e543
Sep 09 12:49:29.207780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.207758    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t"]
Sep 09 12:49:29.209437 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:29.209409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f392a16_e3e2_4bb1_b745_2575ec7debe7.slice/crio-72c3e1650cb80ed092c48318641bab210d23185ad1babb95b6f94df64064e191 WatchSource:0}: Error finding container 72c3e1650cb80ed092c48318641bab210d23185ad1babb95b6f94df64064e191: Status 404 returned error can't find the container with id 72c3e1650cb80ed092c48318641bab210d23185ad1babb95b6f94df64064e191
Sep 09 12:49:29.234629 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.234605    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-b548b666-qdt2z"]
Sep 09 12:49:29.240046 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:29.240010    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3a5edb6_f4ac_4532_8fbe_ac6dc8fbffd3.slice/crio-fe3a05870c8805b88f69b985faf8326de4059288c12ae9a15de0e92af014789f WatchSource:0}: Error finding container fe3a05870c8805b88f69b985faf8326de4059288c12ae9a15de0e92af014789f: Status 404 returned error can't find the container with id fe3a05870c8805b88f69b985faf8326de4059288c12ae9a15de0e92af014789f
Sep 09 12:49:29.252085 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.252021    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz"]
Sep 09 12:49:29.272230 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.272049    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-fj4kt"]
Sep 09 12:49:29.274124 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:29.274095    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcad0a6c1_e143_45f1_a7f3_08826bc0c048.slice/crio-900dffb045828848a42e38d38cb83970e1b597fc4d427c66912afd3fba1c62c9 WatchSource:0}: Error finding container 900dffb045828848a42e38d38cb83970e1b597fc4d427c66912afd3fba1c62c9: Status 404 returned error can't find the container with id 900dffb045828848a42e38d38cb83970e1b597fc4d427c66912afd3fba1c62c9
Sep 09 12:49:29.292550 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.292468    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-zr2l7"]
Sep 09 12:49:29.749737 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.749702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt" event={"ID":"cad0a6c1-e143-45f1-a7f3-08826bc0c048","Type":"ContainerStarted","Data":"900dffb045828848a42e38d38cb83970e1b597fc4d427c66912afd3fba1c62c9"}
Sep 09 12:49:29.751099 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.751077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkb9w" event={"ID":"43acb564-ce77-4108-a5be-23fe3d0ea28e","Type":"ContainerStarted","Data":"d86a1313392a1c1a42003288ea786dba984e57449481ada04a27b2c332ac7921"}
Sep 09 12:49:29.751223 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.751113    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-hkb9w" event={"ID":"43acb564-ce77-4108-a5be-23fe3d0ea28e","Type":"ContainerStarted","Data":"26b1b83dcceea8b8e5dfe3460fe4772340301292fd2a575372bfea3fa166917a"}
Sep 09 12:49:29.751223 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.751177    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:49:29.752193 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.752172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr2l7" event={"ID":"45fa2142-a4b5-405b-80a4-27438f6e9080","Type":"ContainerStarted","Data":"96d3cd53afdaea4bb26af27483120373e34c40159b7c23ada9a1a857094b11aa"}
Sep 09 12:49:29.753248 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.753229    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz" event={"ID":"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6","Type":"ContainerStarted","Data":"e105fea8e5e1dc8301944763d052885f0413f3ae61906c8636aa00e0a147f82e"}
Sep 09 12:49:29.754569 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.754543    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b548b666-qdt2z" event={"ID":"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3","Type":"ContainerStarted","Data":"a033ac1eeeee4a99ac33f6f5730ea046c984e969f8147c0e067fd5c167180206"}
Sep 09 12:49:29.754656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.754584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-b548b666-qdt2z" event={"ID":"e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3","Type":"ContainerStarted","Data":"fe3a05870c8805b88f69b985faf8326de4059288c12ae9a15de0e92af014789f"}
Sep 09 12:49:29.755964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.755941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" event={"ID":"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608","Type":"ContainerStarted","Data":"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"}
Sep 09 12:49:29.756037 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.755979    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" event={"ID":"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608","Type":"ContainerStarted","Data":"5a313b2c17b271464da8567224374592607abfdda0fc7cb2ca88007bfa17e543"}
Sep 09 12:49:29.756037 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.756028    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:29.757079 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.757060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t" event={"ID":"7f392a16-e3e2-4bb1-b745-2575ec7debe7","Type":"ContainerStarted","Data":"72c3e1650cb80ed092c48318641bab210d23185ad1babb95b6f94df64064e191"}
Sep 09 12:49:29.766818 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.766775    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-hkb9w" podStartSLOduration=66.766758418 podStartE2EDuration="1m6.766758418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:29.764870329 +0000 UTC m=+67.800118235" watchObservedRunningTime="2026-09-09 12:49:29.766758418 +0000 UTC m=+67.802006325"
Sep 09 12:49:29.781396 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.781349    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" podStartSLOduration=65.781333083 podStartE2EDuration="1m5.781333083s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:29.78083373 +0000 UTC m=+67.816081630" watchObservedRunningTime="2026-09-09 12:49:29.781333083 +0000 UTC m=+67.816580991"
Sep 09 12:49:29.799291 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.799254    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-b548b666-qdt2z" podStartSLOduration=42.799238143 podStartE2EDuration="42.799238143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:29.79859863 +0000 UTC m=+67.833846538" watchObservedRunningTime="2026-09-09 12:49:29.799238143 +0000 UTC m=+67.834486053"
Sep 09 12:49:29.924478 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.924338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:29.927236 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:29.927180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:30.767378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:30.766826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7pfr" event={"ID":"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799","Type":"ContainerStarted","Data":"d1b6ff4a2f49fed5d2b4ad0bac4039283347ca610f8a4f3a3ce30d8958945c44"}
Sep 09 12:49:30.767378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:30.766893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-q7pfr" event={"ID":"e26aeb9a-4ba3-42ce-9b66-7d8cf6fc3799","Type":"ContainerStarted","Data":"e09429a70bdc775bb2dd04f72d2667a3b5585ed0b6dee55168e6439850985a81"}
Sep 09 12:49:30.767378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:30.766925    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:30.768462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:30.768408    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-b548b666-qdt2z"
Sep 09 12:49:30.790374 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:30.790315    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-q7pfr" podStartSLOduration=66.693136625 podStartE2EDuration="1m7.790295911s" totalImagesPullingTime="1.097159286s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:30.781902028 +0000 UTC m=+68.817149935" watchObservedRunningTime="2026-09-09 12:49:30.790295911 +0000 UTC m=+68.825543819"
Sep 09 12:49:32.917085 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:32.917016    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:33.003755 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:33.003714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:33.103547 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:33.103449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:33.203539 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:33.203491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:34.739513 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.739372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:34.784595 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.784557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t" event={"ID":"7f392a16-e3e2-4bb1-b745-2575ec7debe7","Type":"ContainerStarted","Data":"7bc32a550dc3709a32d38e9e7956d1244abc9fc8445f7885252480bc029a3756"}
Sep 09 12:49:34.786778 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.786557    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt" event={"ID":"cad0a6c1-e143-45f1-a7f3-08826bc0c048","Type":"ContainerStarted","Data":"e28fc0546fb5b8745fc1fbfd3bff80a41dbfa416a5293d4ac2aa91d3093cbf90"}
Sep 09 12:49:34.790432 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.790407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr2l7" event={"ID":"45fa2142-a4b5-405b-80a4-27438f6e9080","Type":"ContainerStarted","Data":"dda95457d2db8b4d0c676d945ffe2e3358963c0dad36e72dc122105d3f6d19ef"}
Sep 09 12:49:34.790537 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.790458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-zr2l7" event={"ID":"45fa2142-a4b5-405b-80a4-27438f6e9080","Type":"ContainerStarted","Data":"aac9ad626d55b146481c3fdbe6ca45ed5f69dc36cad4898e449196a629ca1267"}
Sep 09 12:49:34.792319 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.792289    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz" event={"ID":"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6","Type":"ContainerStarted","Data":"527ebac9a50536efe9d636c91ebf1eecafc6bfce05976e1a4248f0377d88e61e"}
Sep 09 12:49:34.792415 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.792338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz" event={"ID":"a163c869-c9d7-42cf-a8cc-8c7aaf2594b6","Type":"ContainerStarted","Data":"7e6efb6160430732436c76f739de525a36367552e4e7ea274a885aa51c51f6b0"}
Sep 09 12:49:34.799617 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.799578    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6cdcc96965-9kn6t" podStartSLOduration=42.71307346 podStartE2EDuration="47.799565792s" totalImagesPullingTime="5.086492332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:34.799126975 +0000 UTC m=+72.834374883" watchObservedRunningTime="2026-09-09 12:49:34.799565792 +0000 UTC m=+72.834813698"
Sep 09 12:49:34.812801 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.812764    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-fj4kt" podStartSLOduration=34.787166039 podStartE2EDuration="39.812745097s" totalImagesPullingTime="5.025579058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:34.812537519 +0000 UTC m=+72.847785428" watchObservedRunningTime="2026-09-09 12:49:34.812745097 +0000 UTC m=+72.847993005"
Sep 09 12:49:34.818259 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.818234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:34.832093 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.832043    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-7cdcfcf76c-84nxz" podStartSLOduration=42.871469838 podStartE2EDuration="47.832025235s" totalImagesPullingTime="4.960555397s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:34.830639555 +0000 UTC m=+72.865887456" watchObservedRunningTime="2026-09-09 12:49:34.832025235 +0000 UTC m=+72.867273145"
Sep 09 12:49:34.939429 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:34.939334    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.014865 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.014830    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.114840 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.114805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.179992 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.179961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.414134 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.414087    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.513900 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.513870    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.571807 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.571783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:35.797226 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.797155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-fj9l6" event={"ID":"abd06375-8c1e-49ec-9a04-9782206cadbc","Type":"ContainerStarted","Data":"f8d92507166c2b716c7fab2d293e484985ca55cb81abbd771b438eb9a7f87949"}
Sep 09 12:49:35.812865 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.812822    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-fj9l6" podStartSLOduration=66.807345211 podStartE2EDuration="1m12.812805742s" totalImagesPullingTime="6.005460531s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 12:48:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:35.812576303 +0000 UTC m=+73.847824210" watchObservedRunningTime="2026-09-09 12:49:35.812805742 +0000 UTC m=+73.848053649"
Sep 09 12:49:35.829645 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:35.829607    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-zr2l7" podStartSLOduration=34.829856601 podStartE2EDuration="39.82959401s" totalImagesPullingTime="4.999737409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:35.828616957 +0000 UTC m=+73.863864877" watchObservedRunningTime="2026-09-09 12:49:35.82959401 +0000 UTC m=+73.864842000"
Sep 09 12:49:36.873138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:36.873109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:36.973692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:36.973660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:38.797928 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:38.797894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:39.944197 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:39.944158    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:40.012004 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:40.011971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:40.467980 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:40.467946    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:41.447066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.447039    2580 scope.go:117] "RemoveContainer" containerID="34f589df0c89427c85d6e0d1a678200a22acea6ebfcb9077eb567c08fba649a6"
Sep 09 12:49:41.820680 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.820607    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:49:41.820814 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.820689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" event={"ID":"42bd9e84-8866-4e8c-a958-a96cd635510b","Type":"ContainerStarted","Data":"187c5cf62e804c126916c0cff9c9ee45b39ff311474c29353d15df6c0b7d8a1f"}
Sep 09 12:49:41.820982 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.820963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:41.837865 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.837818    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-cg7vj" podStartSLOduration=46.420910139 podStartE2EDuration="54.837804933s" totalImagesPullingTime="8.416894794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:48:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:41.836919686 +0000 UTC m=+79.872167593" watchObservedRunningTime="2026-09-09 12:49:41.837804933 +0000 UTC m=+79.873052840"
Sep 09 12:49:41.949634 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:41.949600    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:42.047728 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:42.047696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:42.161901 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:42.161868    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:42.803304 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:42.803278    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-cg7vj"
Sep 09 12:49:43.039377 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.039337    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:43.103645 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.103563    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:43.369080 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.368997    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:43.499840 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.499808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:43.600681 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.600649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:43.700303 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:43.700277    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.050842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:44.050768    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.110695 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:44.110667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.666088 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:44.666059    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:44.800313 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:44.800208    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-zr2l7"
Sep 09 12:49:44.866563 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:44.866494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:45.019190 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.019116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:45.122513 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.122481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:45.220660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.220633    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:45.363714 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.363638    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"]
Sep 09 12:49:45.366381 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.366366    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.369070 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.369050    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-qxswr\""
Sep 09 12:49:45.369334 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.369319    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 09 12:49:45.370007 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.369993    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 09 12:49:45.370095 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.369996    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 12:49:45.370156 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.370115    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 09 12:49:45.373529 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.373490    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-64hhn"]
Sep 09 12:49:45.375763 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.375747    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.377887 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.377868    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 12:49:45.377996 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.377928    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t4lg7\""
Sep 09 12:49:45.378252 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.378227    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 12:49:45.378343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.378239    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 12:49:45.379937 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.379917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"]
Sep 09 12:49:45.443082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-root\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443087    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-metrics-client-ca\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-accelerators-collector-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443126    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443275    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acab4b9d-26c3-466d-b619-72409878a2e5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443300    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-sys\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443339    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-wtmp\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443355    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk5qh\" (UniqueName: \"kubernetes.io/projected/acab4b9d-26c3-466d-b619-72409878a2e5-kube-api-access-mk5qh\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.443484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443406    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbscv\" (UniqueName: \"kubernetes.io/projected/4619d408-59d1-4479-bdbc-569b9c104531-kube-api-access-lbscv\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.443484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.443427    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-textfile\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543703    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acab4b9d-26c3-466d-b619-72409878a2e5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.543729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543732    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-sys\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543750    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-wtmp\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543811    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-sys\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:45.543843    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk5qh\" (UniqueName: \"kubernetes.io/projected/acab4b9d-26c3-466d-b619-72409878a2e5-kube-api-access-mk5qh\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:45.543904    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls podName:4619d408-59d1-4479-bdbc-569b9c104531 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:46.043883386 +0000 UTC m=+84.079131271 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls") pod "node-exporter-64hhn" (UID: "4619d408-59d1-4479-bdbc-569b9c104531") : secret "node-exporter-tls" not found
Sep 09 12:49:45.543939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.543922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-wtmp\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544043    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbscv\" (UniqueName: \"kubernetes.io/projected/4619d408-59d1-4479-bdbc-569b9c104531-kube-api-access-lbscv\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-textfile\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-root\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544145    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/acab4b9d-26c3-466d-b619-72409878a2e5-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544157    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-metrics-client-ca\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-accelerators-collector-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4619d408-59d1-4479-bdbc-569b9c104531-root\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544234    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.544326 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.544961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:45.544332    2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 09 12:49:45.544961 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:49:45.544419    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls podName:acab4b9d-26c3-466d-b619-72409878a2e5 nodeName:}" failed. No retries permitted until 2026-09-09 12:49:46.044403305 +0000 UTC m=+84.079651190 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-nnm6m" (UID: "acab4b9d-26c3-466d-b619-72409878a2e5") : secret "kube-state-metrics-tls" not found
Sep 09 12:49:45.544961 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544530    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-textfile\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.544961 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.544879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-accelerators-collector-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.545149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.545025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.545149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.545072    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.545221 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.545177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4619d408-59d1-4479-bdbc-569b9c104531-metrics-client-ca\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.546230 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.546213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:45.546674 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.546657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.552424 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.552400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk5qh\" (UniqueName: \"kubernetes.io/projected/acab4b9d-26c3-466d-b619-72409878a2e5-kube-api-access-mk5qh\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:45.552856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:45.552761    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbscv\" (UniqueName: \"kubernetes.io/projected/4619d408-59d1-4479-bdbc-569b9c104531-kube-api-access-lbscv\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:46.048590 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.048554    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:46.048772 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.048623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:46.050928 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.050901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4619d408-59d1-4479-bdbc-569b9c104531-node-exporter-tls\") pod \"node-exporter-64hhn\" (UID: \"4619d408-59d1-4479-bdbc-569b9c104531\") " pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:46.051030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.050931    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/acab4b9d-26c3-466d-b619-72409878a2e5-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-nnm6m\" (UID: \"acab4b9d-26c3-466d-b619-72409878a2e5\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:46.276343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.276307    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"
Sep 09 12:49:46.283771 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.283748    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-64hhn"
Sep 09 12:49:46.294888 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:46.294850    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4619d408_59d1_4479_bdbc_569b9c104531.slice/crio-64a6935ca9e9a823cc3393edbb01080b3a9afc5b9b8fa4c93c698f0785b50a61 WatchSource:0}: Error finding container 64a6935ca9e9a823cc3393edbb01080b3a9afc5b9b8fa4c93c698f0785b50a61: Status 404 returned error can't find the container with id 64a6935ca9e9a823cc3393edbb01080b3a9afc5b9b8fa4c93c698f0785b50a61
Sep 09 12:49:46.327061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.327027    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:46.419325 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.419275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:46.555692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.555613    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:46.575246 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.575220    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m"]
Sep 09 12:49:46.579579 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:46.579556    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacab4b9d_26c3_466d_b619_72409878a2e5.slice/crio-e086386506c8eb3002509463b9656d0d20c3b78b6a8ae21c25c34c043de02d1b WatchSource:0}: Error finding container e086386506c8eb3002509463b9656d0d20c3b78b6a8ae21c25c34c043de02d1b: Status 404 returned error can't find the container with id e086386506c8eb3002509463b9656d0d20c3b78b6a8ae21c25c34c043de02d1b
Sep 09 12:49:46.848708 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.848620    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m" event={"ID":"acab4b9d-26c3-466d-b619-72409878a2e5","Type":"ContainerStarted","Data":"e086386506c8eb3002509463b9656d0d20c3b78b6a8ae21c25c34c043de02d1b"}
Sep 09 12:49:46.849894 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:46.849874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64hhn" event={"ID":"4619d408-59d1-4479-bdbc-569b9c104531","Type":"ContainerStarted","Data":"64a6935ca9e9a823cc3393edbb01080b3a9afc5b9b8fa4c93c698f0785b50a61"}
Sep 09 12:49:47.194678 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:47.194642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:48.860482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.860392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m" event={"ID":"acab4b9d-26c3-466d-b619-72409878a2e5","Type":"ContainerStarted","Data":"7d2b89e027c3f5fb8fdf1de4d2027166617eafd71d4c06dca9cd5b711a62a730"}
Sep 09 12:49:48.860482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.860456    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m" event={"ID":"acab4b9d-26c3-466d-b619-72409878a2e5","Type":"ContainerStarted","Data":"cbd7a4d4c72cd961e8be9caddf02445576068082e1ac64bfe58d81445616b438"}
Sep 09 12:49:48.860482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.860483    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m" event={"ID":"acab4b9d-26c3-466d-b619-72409878a2e5","Type":"ContainerStarted","Data":"45e6a89ef72c7a67213e480b7a2bee4d39e9abc16cb3ea71405beb0c66ddad53"}
Sep 09 12:49:48.862151 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.862122    2580 generic.go:356] "Generic (PLEG): container finished" podID="4619d408-59d1-4479-bdbc-569b9c104531" containerID="4e6fa5a1fdac7c19f49238cefb83de84cfce239673522dacbae7aa0f7e0b5924" exitCode=0
Sep 09 12:49:48.862261 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.862201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64hhn" event={"ID":"4619d408-59d1-4479-bdbc-569b9c104531","Type":"ContainerDied","Data":"4e6fa5a1fdac7c19f49238cefb83de84cfce239673522dacbae7aa0f7e0b5924"}
Sep 09 12:49:48.878359 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:48.878320    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-nnm6m" podStartSLOduration=2.5310679 podStartE2EDuration="3.878306244s" totalImagesPullingTime="1.347238344s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:48.877132109 +0000 UTC m=+86.912380017" watchObservedRunningTime="2026-09-09 12:49:48.878306244 +0000 UTC m=+86.913554148"
Sep 09 12:49:49.259926 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.259898    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:49.359240 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.359206    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:49.417429 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.417396    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:49.867869 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.867837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64hhn" event={"ID":"4619d408-59d1-4479-bdbc-569b9c104531","Type":"ContainerStarted","Data":"e067be3029f313adf9585a28349ffc0049ea5e17fbc6c0b043d6272b9081e342"}
Sep 09 12:49:49.868322 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.867920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-64hhn" event={"ID":"4619d408-59d1-4479-bdbc-569b9c104531","Type":"ContainerStarted","Data":"32eeb4e29c6ce89fd10b02a5549bb38ceccd70c78fc2fa6be814e99fcb95ada0"}
Sep 09 12:49:49.900304 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:49.900262    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-64hhn" podStartSLOduration=2.60702444 podStartE2EDuration="4.900248771s" totalImagesPullingTime="2.231236942s" totalInitContainerRuntime="61.987389ms" isStatefulPod=true podCreationTimestamp="2026-09-09 12:49:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:49.899155731 +0000 UTC m=+87.934403651" watchObservedRunningTime="2026-09-09 12:49:49.900248771 +0000 UTC m=+87.935496678"
Sep 09 12:49:50.113555 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.113525    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"]
Sep 09 12:49:50.115475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.115457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.117572 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.117551    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 12:49:50.117653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.117570    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-s6w9r\""
Sep 09 12:49:50.123869 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.123845    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"]
Sep 09 12:49:50.157170 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.157148    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:50.183295 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.183273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/176a869c-96ea-4c00-9758-3b5c06c8e4ef-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.183370 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.183327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gw9mg\" (UniqueName: \"kubernetes.io/projected/176a869c-96ea-4c00-9758-3b5c06c8e4ef-kube-api-access-gw9mg\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.234617 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.234594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:50.284265 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.284242    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gw9mg\" (UniqueName: \"kubernetes.io/projected/176a869c-96ea-4c00-9758-3b5c06c8e4ef-kube-api-access-gw9mg\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.284361 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.284280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/176a869c-96ea-4c00-9758-3b5c06c8e4ef-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.286720 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.286698    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/176a869c-96ea-4c00-9758-3b5c06c8e4ef-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.291945 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.291888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gw9mg\" (UniqueName: \"kubernetes.io/projected/176a869c-96ea-4c00-9758-3b5c06c8e4ef-kube-api-access-gw9mg\") pod \"monitoring-plugin-85bbbbbf4-z8l7h\" (UID: \"176a869c-96ea-4c00-9758-3b5c06c8e4ef\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.291945 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.291932    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:50.424898 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.424871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:50.690108 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.689952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"]
Sep 09 12:49:50.692027 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:49:50.691988    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod176a869c_96ea_4c00_9758_3b5c06c8e4ef.slice/crio-0697b14ee67db95e8b55fa01eabb03186142693b69ef753dfcb64bc0837013f9 WatchSource:0}: Error finding container 0697b14ee67db95e8b55fa01eabb03186142693b69ef753dfcb64bc0837013f9: Status 404 returned error can't find the container with id 0697b14ee67db95e8b55fa01eabb03186142693b69ef753dfcb64bc0837013f9
Sep 09 12:49:50.771568 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.771548    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:49:50.872780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:50.872746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h" event={"ID":"176a869c-96ea-4c00-9758-3b5c06c8e4ef","Type":"ContainerStarted","Data":"0697b14ee67db95e8b55fa01eabb03186142693b69ef753dfcb64bc0837013f9"}
Sep 09 12:49:51.924066 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:51.924030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:51.981359 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:51.981336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:53.885986 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:53.885952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h" event={"ID":"176a869c-96ea-4c00-9758-3b5c06c8e4ef","Type":"ContainerStarted","Data":"923907d58943deb504165a13ef3089a344d8c926fe6ea1d69691b50fecf31af2"}
Sep 09 12:49:53.886361 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:53.886145    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:53.892294 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:53.892263    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h"
Sep 09 12:49:53.904067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:53.904026    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-z8l7h" podStartSLOduration=1.491126629 podStartE2EDuration="3.904012362s" totalImagesPullingTime="2.412885733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:49:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:49:53.90252978 +0000 UTC m=+91.937777686" watchObservedRunningTime="2026-09-09 12:49:53.904012362 +0000 UTC m=+91.939260310"
Sep 09 12:49:54.915819 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:54.915784    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:49:54.929622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:54.929590    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.028676 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:55.028639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:55.129772 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:55.129737    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:57.810520 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:57.810469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:57.910172 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:57.910133    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:58.122338 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:58.122272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:49:58.193404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:49:58.193373    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:00.121409 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:00.121370    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:00.177605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:00.177575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:00.769597 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:00.769568    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-hkb9w"
Sep 09 12:50:01.681964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.681934    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:50:01.686913 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.686892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.689592 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.689572    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 12:50:01.690512 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690479    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 12:50:01.690624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690542    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 12:50:01.690624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690564    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 12:50:01.690624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690479    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-2vjnl\""
Sep 09 12:50:01.690624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690610    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 12:50:01.690839 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690482    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 12:50:01.690839 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.690687    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 12:50:01.694723 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.694705    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 12:50:01.698464 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.698445    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:50:01.709541 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.709515    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:01.769817 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769792    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.769935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwmrp\" (UniqueName: \"kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.769935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769846    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.769935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.770037 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.770037 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.770037 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.769986    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.810325 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.810278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:01.870383 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870354    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwmrp\" (UniqueName: \"kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870383 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870465    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870479    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.870535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.870518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.871194 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.871170    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.871194 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.871184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.871363 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.871175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.871425 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.871371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.872977 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.872958    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.873074 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.873012    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.878686 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.878662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwmrp\" (UniqueName: \"kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp\") pod \"console-b6776d99f-7tcrs\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") " pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:01.996247 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:01.996189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:02.269316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:02.269292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:50:02.271773 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:50:02.271744    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14836304_294a_4cfd_b296_8e47740b88c1.slice/crio-bae392166e2d0ec902dd1e296727dcd44956d5a4cef3f777f8b76035b7dbce6e WatchSource:0}: Error finding container bae392166e2d0ec902dd1e296727dcd44956d5a4cef3f777f8b76035b7dbce6e: Status 404 returned error can't find the container with id bae392166e2d0ec902dd1e296727dcd44956d5a4cef3f777f8b76035b7dbce6e
Sep 09 12:50:02.920376 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:02.920340    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6776d99f-7tcrs" event={"ID":"14836304-294a-4cfd-b296-8e47740b88c1","Type":"ContainerStarted","Data":"bae392166e2d0ec902dd1e296727dcd44956d5a4cef3f777f8b76035b7dbce6e"}
Sep 09 12:50:03.456868 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:03.456831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:03.550215 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:03.550157    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:05.544343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:05.544304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:05.601177 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:05.601142    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:05.934902 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:05.934870    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6776d99f-7tcrs" event={"ID":"14836304-294a-4cfd-b296-8e47740b88c1","Type":"ContainerStarted","Data":"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"}
Sep 09 12:50:05.952313 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:05.952262    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b6776d99f-7tcrs" podStartSLOduration=1.479555193 podStartE2EDuration="4.952248086s" totalImagesPullingTime="3.472692893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:50:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:50:05.951123521 +0000 UTC m=+103.986371429" watchObservedRunningTime="2026-09-09 12:50:05.952248086 +0000 UTC m=+103.987495994"
Sep 09 12:50:06.893702 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:06.893666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:06.997946 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:06.997908    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:10.294848 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:10.294805    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:10.350892 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:10.350860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:11.959149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.959116    2580 generic.go:356] "Generic (PLEG): container finished" podID="5f92c179-25c7-4c7a-89fb-286028524d9f" containerID="6681605d8a1cd07b357d8847aa67e5b6318409b6857cd4e6dc49ace3921365ad" exitCode=0
Sep 09 12:50:11.959589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.959188    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb" event={"ID":"5f92c179-25c7-4c7a-89fb-286028524d9f","Type":"ContainerDied","Data":"6681605d8a1cd07b357d8847aa67e5b6318409b6857cd4e6dc49ace3921365ad"}
Sep 09 12:50:11.959589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.959536    2580 scope.go:117] "RemoveContainer" containerID="6681605d8a1cd07b357d8847aa67e5b6318409b6857cd4e6dc49ace3921365ad"
Sep 09 12:50:11.960845 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.960828    2580 generic.go:356] "Generic (PLEG): container finished" podID="5613ea48-561a-49c6-84da-94fa313b1668" containerID="a63e88bd06ddceedeac58e0dd7b4fbb7a19c26d9638e0ec17c4a987f687a6dde" exitCode=0
Sep 09 12:50:11.960906 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.960890    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9" event={"ID":"5613ea48-561a-49c6-84da-94fa313b1668","Type":"ContainerDied","Data":"a63e88bd06ddceedeac58e0dd7b4fbb7a19c26d9638e0ec17c4a987f687a6dde"}
Sep 09 12:50:11.961177 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:11.961161    2580 scope.go:117] "RemoveContainer" containerID="a63e88bd06ddceedeac58e0dd7b4fbb7a19c26d9638e0ec17c4a987f687a6dde"
Sep 09 12:50:12.007524 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.002256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:12.007524 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.004283    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:12.017605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.017581    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:12.966740 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.966702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-5zfcb" event={"ID":"5f92c179-25c7-4c7a-89fb-286028524d9f","Type":"ContainerStarted","Data":"18fea4d05ac2b33d0af8fde2486018da6ebb1953e67a2b2d72bd3a2440a985dd"}
Sep 09 12:50:12.968493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.968469    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-5gsr9" event={"ID":"5613ea48-561a-49c6-84da-94fa313b1668","Type":"ContainerStarted","Data":"6caf9b9fc2c9e9c9586bce497394d6df50c1434cb17a9151d0791fd6cbe4c268"}
Sep 09 12:50:12.972378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:12.972356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:50:13.139529 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:13.139488    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:13.189551 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:13.189493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:13.243871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:13.243788    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:19.935473 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:19.935413    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" podUID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" containerName="registry" containerID="cri-o://ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490" gracePeriod=30
Sep 09 12:50:20.174580 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.174559    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:50:20.323574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323467    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323574 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323565    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323598    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vfmrp\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323674    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.323842 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.323727    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates\") pod \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\" (UID: \"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608\") "
Sep 09 12:50:20.324181 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.324127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:20.324311 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.324254    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:50:20.326132 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.326106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:20.326244 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.326202    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:20.326306 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.326265    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:20.326349 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.326309    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp" (OuterVolumeSpecName: "kube-api-access-vfmrp") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "kube-api-access-vfmrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:50:20.326414 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.326396    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:50:20.332111 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.332083    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" (UID: "dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:50:20.424984 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.424943    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-image-registry-private-configuration\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.424984 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.424982    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-trusted-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.424998    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-bound-sa-token\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.425012    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vfmrp\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-kube-api-access-vfmrp\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.425024    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-tls\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.425035    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-installation-pull-secrets\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.425046    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-ca-trust-extracted\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.425143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.425057    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608-registry-certificates\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:50:20.541823 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.541792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:20.641587 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.641564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:20.999605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.999530    2580 generic.go:356] "Generic (PLEG): container finished" podID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" containerID="ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490" exitCode=0
Sep 09 12:50:20.999605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.999568    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"
Sep 09 12:50:20.999605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.999594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" event={"ID":"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608","Type":"ContainerDied","Data":"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"}
Sep 09 12:50:21.000068 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.999647    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6c88ccbb9b-zjlhw" event={"ID":"dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608","Type":"ContainerDied","Data":"5a313b2c17b271464da8567224374592607abfdda0fc7cb2ca88007bfa17e543"}
Sep 09 12:50:21.000068 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:20.999668    2580 scope.go:117] "RemoveContainer" containerID="ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"
Sep 09 12:50:21.007859 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:21.007844    2580 scope.go:117] "RemoveContainer" containerID="ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"
Sep 09 12:50:21.008131 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:50:21.008109    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490\": container with ID starting with ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490 not found: ID does not exist" containerID="ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"
Sep 09 12:50:21.008178 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:21.008139    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490"} err="failed to get container status \"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490\": rpc error: code = NotFound desc = could not find container \"ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490\": container with ID starting with ed143a01b27520a4dc2e738fb8723750e44bbf91ecc0c28f7cba495d62871490 not found: ID does not exist"
Sep 09 12:50:21.015813 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:21.015792    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:50:21.021399 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:21.021378    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6c88ccbb9b-zjlhw"]
Sep 09 12:50:22.451756 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:22.451724    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" path="/var/lib/kubelet/pods/dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608/volumes"
Sep 09 12:50:27.025738 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:27.025657    2580 generic.go:356] "Generic (PLEG): container finished" podID="66e4f7f3-6970-4189-b923-865fbac766d2" containerID="98e9bf69a1fbdca1ec37d3714898a68fa83deec9cc576526bc5ff6d3bccb198a" exitCode=0
Sep 09 12:50:27.026090 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:27.025730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd" event={"ID":"66e4f7f3-6970-4189-b923-865fbac766d2","Type":"ContainerDied","Data":"98e9bf69a1fbdca1ec37d3714898a68fa83deec9cc576526bc5ff6d3bccb198a"}
Sep 09 12:50:27.026090 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:27.026063    2580 scope.go:117] "RemoveContainer" containerID="98e9bf69a1fbdca1ec37d3714898a68fa83deec9cc576526bc5ff6d3bccb198a"
Sep 09 12:50:27.660577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:27.660550    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b548b666-qdt2z_e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3/router/0.log"
Sep 09 12:50:27.668313 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:27.668286    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l9tv_6153e58f-438f-44f4-a984-421cae427aea/serve-healthcheck-canary/0.log"
Sep 09 12:50:28.031737 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:28.031648    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-rp5wd" event={"ID":"66e4f7f3-6970-4189-b923-865fbac766d2","Type":"ContainerStarted","Data":"a96f97e5c7c1c2760397f0eb29980e05b38db34e566a11a4795dbb6e150d24d3"}
Sep 09 12:50:28.641256 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:28.641223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:50:28.741034 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:50:28.740991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:04.559813 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:04.559775    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:04.884281 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:04.884195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:12.482468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.482437    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:51:12.483041 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.482948    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" containerName="registry"
Sep 09 12:51:12.483041 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.482969    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" containerName="registry"
Sep 09 12:51:12.483166 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.483067    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dfdef752-22f2-4bf6-a3a0-fc9f9bfb3608" containerName="registry"
Sep 09 12:51:12.488125 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.487966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.495995 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.495972    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:51:12.542798 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.542767    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:12.602368 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602340    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:12.602626 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602683 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602683 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602748 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602681    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602783 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602824 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.602824 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.602814    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgrmg\" (UniqueName: \"kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703708 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgrmg\" (UniqueName: \"kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703714    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703733    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703889 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703868    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703934 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703921    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.703997 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.703982    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.704048 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.704012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.704448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.704426    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.704567 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.704545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.704627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.704595    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.704627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.704610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.706408 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.706391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.706596 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.706580    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.711774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.711746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgrmg\" (UniqueName: \"kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg\") pod \"console-85475bcc65-h5jt4\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") " pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:12.798804 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:12.798755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:13.067902 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:13.067830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:51:13.070915 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:51:13.070885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b1098bd_4cc5_45d8_9993_399ca33cfbe3.slice/crio-9171bb06638fc5de6fd5ecd24d5e7de50f0c2721af50f0e5816268985cc5f490 WatchSource:0}: Error finding container 9171bb06638fc5de6fd5ecd24d5e7de50f0c2721af50f0e5816268985cc5f490: Status 404 returned error can't find the container with id 9171bb06638fc5de6fd5ecd24d5e7de50f0c2721af50f0e5816268985cc5f490
Sep 09 12:51:13.206756 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:13.206719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85475bcc65-h5jt4" event={"ID":"1b1098bd-4cc5-45d8-9993-399ca33cfbe3","Type":"ContainerStarted","Data":"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"}
Sep 09 12:51:13.206933 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:13.206772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85475bcc65-h5jt4" event={"ID":"1b1098bd-4cc5-45d8-9993-399ca33cfbe3","Type":"ContainerStarted","Data":"9171bb06638fc5de6fd5ecd24d5e7de50f0c2721af50f0e5816268985cc5f490"}
Sep 09 12:51:13.224017 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:13.223970    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85475bcc65-h5jt4" podStartSLOduration=1.223953753 podStartE2EDuration="1.223953753s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:51:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:51:13.222094273 +0000 UTC m=+171.257342180" watchObservedRunningTime="2026-09-09 12:51:13.223953753 +0000 UTC m=+171.259201660"
Sep 09 12:51:22.799456 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:22.799370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:22.799456 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:22.799434    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:22.804058 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:22.804035    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:23.249069 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:23.249045    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:51:23.291608 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:23.291579    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:51:23.394468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:23.394437    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:23.442911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:23.442869    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:23.497382 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:23.497341    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:48.314105 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.314056    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b6776d99f-7tcrs" podUID="14836304-294a-4cfd-b296-8e47740b88c1" containerName="console" containerID="cri-o://f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a" gracePeriod=15
Sep 09 12:51:48.553631 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.553606    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:51:48.677485 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677672 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677491    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677672 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677527    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677672 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677557    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lwmrp\" (UniqueName: \"kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677672 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677735    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.677885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.677829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config\") pod \"14836304-294a-4cfd-b296-8e47740b88c1\" (UID: \"14836304-294a-4cfd-b296-8e47740b88c1\") "
Sep 09 12:51:48.678097 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.678059    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:51:48.678167 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.678065    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:51:48.678167 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.678114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca" (OuterVolumeSpecName: "service-ca") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:51:48.678373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.678350    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config" (OuterVolumeSpecName: "console-config") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:51:48.679915 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.679893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp" (OuterVolumeSpecName: "kube-api-access-lwmrp") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "kube-api-access-lwmrp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:51:48.680226 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.680198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:51:48.680319 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.680220    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "14836304-294a-4cfd-b296-8e47740b88c1" (UID: "14836304-294a-4cfd-b296-8e47740b88c1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:51:48.778745 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778717    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-trusted-ca-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778745 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778742    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-oauth-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778745 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778753    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-oauth-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778761    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwmrp\" (UniqueName: \"kubernetes.io/projected/14836304-294a-4cfd-b296-8e47740b88c1-kube-api-access-lwmrp\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778771    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/14836304-294a-4cfd-b296-8e47740b88c1-console-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778779    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:48.778976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:48.778788    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/14836304-294a-4cfd-b296-8e47740b88c1-console-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:51:49.352753 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.352719    2580 generic.go:356] "Generic (PLEG): container finished" podID="14836304-294a-4cfd-b296-8e47740b88c1" containerID="f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a" exitCode=0
Sep 09 12:51:49.353162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.352765    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b6776d99f-7tcrs"
Sep 09 12:51:49.353162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.352782    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6776d99f-7tcrs" event={"ID":"14836304-294a-4cfd-b296-8e47740b88c1","Type":"ContainerDied","Data":"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"}
Sep 09 12:51:49.353162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.352823    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b6776d99f-7tcrs" event={"ID":"14836304-294a-4cfd-b296-8e47740b88c1","Type":"ContainerDied","Data":"bae392166e2d0ec902dd1e296727dcd44956d5a4cef3f777f8b76035b7dbce6e"}
Sep 09 12:51:49.353162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.352842    2580 scope.go:117] "RemoveContainer" containerID="f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"
Sep 09 12:51:49.361175 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.361159    2580 scope.go:117] "RemoveContainer" containerID="f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"
Sep 09 12:51:49.361434 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:51:49.361411    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a\": container with ID starting with f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a not found: ID does not exist" containerID="f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"
Sep 09 12:51:49.361495 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.361442    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a"} err="failed to get container status \"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a\": rpc error: code = NotFound desc = could not find container \"f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a\": container with ID starting with f2d29fae59be0f93d0e8961cc85c045c8ad544a14b605c496ba66da18a14b17a not found: ID does not exist"
Sep 09 12:51:49.372831 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.372811    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:51:49.376341 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:49.376322    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b6776d99f-7tcrs"]
Sep 09 12:51:50.450956 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:50.450922    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14836304-294a-4cfd-b296-8e47740b88c1" path="/var/lib/kubelet/pods/14836304-294a-4cfd-b296-8e47740b88c1/volumes"
Sep 09 12:51:57.766402 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:57.766362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:51:57.816337 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:51:57.816299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:00.125215 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:00.125179    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:19.779009 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:19.778971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:19.838611 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:19.838567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:30.860914 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:30.860879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:31.758465 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:31.758435    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:31.817833 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:31.817799    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:33.418057 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.418021    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:52:33.418445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.418366    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14836304-294a-4cfd-b296-8e47740b88c1" containerName="console"
Sep 09 12:52:33.418445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.418379    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="14836304-294a-4cfd-b296-8e47740b88c1" containerName="console"
Sep 09 12:52:33.418445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.418437    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="14836304-294a-4cfd-b296-8e47740b88c1" containerName="console"
Sep 09 12:52:33.422665 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.422638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.433643 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.433615    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:52:33.474962 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.474931    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:33.491825 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491843    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491882    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9vn6\" (UniqueName: \"kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491945    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.491970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.491961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.531165 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.531136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:33.592413 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592413 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592533    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592564    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592585    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9vn6\" (UniqueName: \"kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.592630 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.592612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.593193 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.593165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.593300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.593218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.593300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.593284    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.593404 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.593385    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.595100 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.595071    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.595189 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.595133    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.600311 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.600287    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9vn6\" (UniqueName: \"kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6\") pod \"console-7b9fd88b7f-6zpzn\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") " pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:33.613991 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.613966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:33.674916 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.674841    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:33.732522 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:33.732475    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:34.005234 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:34.005142    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:52:34.008970 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:52:34.008945    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c200d1_6dd4_4d0e_813c_3fa0f567667a.slice/crio-e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075 WatchSource:0}: Error finding container e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075: Status 404 returned error can't find the container with id e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075
Sep 09 12:52:34.216443 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:34.216409    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:34.523585 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:34.523538    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd88b7f-6zpzn" event={"ID":"e3c200d1-6dd4-4d0e-813c-3fa0f567667a","Type":"ContainerStarted","Data":"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"}
Sep 09 12:52:34.523585 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:34.523588    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd88b7f-6zpzn" event={"ID":"e3c200d1-6dd4-4d0e-813c-3fa0f567667a","Type":"ContainerStarted","Data":"e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075"}
Sep 09 12:52:34.539424 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:34.539379    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b9fd88b7f-6zpzn" podStartSLOduration=1.539361625 podStartE2EDuration="1.539361625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:52:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:52:34.539235366 +0000 UTC m=+252.574483275" watchObservedRunningTime="2026-09-09 12:52:34.539361625 +0000 UTC m=+252.574609533"
Sep 09 12:52:35.549542 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:35.549492    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:35.650403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:35.650359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:43.732593 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:43.732551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:43.733112 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:43.732612    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:43.736967 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:43.736945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:44.565322 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:44.565292    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:52:44.620413 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:44.620378    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:52:44.750438 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:44.750397    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:44.850257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:44.850156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:44.951262 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:44.951227    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:57.020693 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:57.020626    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:52:57.118586 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:52:57.118542    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:09.645599 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.645555    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85475bcc65-h5jt4" podUID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" containerName="console" containerID="cri-o://569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163" gracePeriod=15
Sep 09 12:53:09.888261 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.888236    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:53:09.952637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952607    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952628    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952677    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952704    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952773    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wgrmg\" (UniqueName: \"kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.952856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952800    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config\") pod \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\" (UID: \"1b1098bd-4cc5-45d8-9993-399ca33cfbe3\") "
Sep 09 12:53:09.953056 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.952956    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config" (OuterVolumeSpecName: "console-config") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:53:09.953056 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.953051    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:09.953204 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.953176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca" (OuterVolumeSpecName: "service-ca") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:53:09.953259 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.953185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:53:09.953259 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.953207    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:53:09.954734 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.954710    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:53:09.955042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.955020    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg" (OuterVolumeSpecName: "kube-api-access-wgrmg") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "kube-api-access-wgrmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:53:09.955042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:09.955034    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1b1098bd-4cc5-45d8-9993-399ca33cfbe3" (UID: "1b1098bd-4cc5-45d8-9993-399ca33cfbe3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:53:10.053579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053535    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wgrmg\" (UniqueName: \"kubernetes.io/projected/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-kube-api-access-wgrmg\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.053579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053573    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-oauth-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.053579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053585    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-console-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.053579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053595    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-trusted-ca-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.053885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053603    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.053885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.053611    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1b1098bd-4cc5-45d8-9993-399ca33cfbe3-oauth-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:53:10.667407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.667376    2580 generic.go:356] "Generic (PLEG): container finished" podID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" containerID="569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163" exitCode=0
Sep 09 12:53:10.667770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.667414    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85475bcc65-h5jt4"
Sep 09 12:53:10.667770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.667460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85475bcc65-h5jt4" event={"ID":"1b1098bd-4cc5-45d8-9993-399ca33cfbe3","Type":"ContainerDied","Data":"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"}
Sep 09 12:53:10.667770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.667519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85475bcc65-h5jt4" event={"ID":"1b1098bd-4cc5-45d8-9993-399ca33cfbe3","Type":"ContainerDied","Data":"9171bb06638fc5de6fd5ecd24d5e7de50f0c2721af50f0e5816268985cc5f490"}
Sep 09 12:53:10.667770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.667540    2580 scope.go:117] "RemoveContainer" containerID="569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"
Sep 09 12:53:10.675121 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.675104    2580 scope.go:117] "RemoveContainer" containerID="569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"
Sep 09 12:53:10.675395 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:53:10.675378    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163\": container with ID starting with 569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163 not found: ID does not exist" containerID="569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"
Sep 09 12:53:10.675450 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.675404    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163"} err="failed to get container status \"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163\": rpc error: code = NotFound desc = could not find container \"569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163\": container with ID starting with 569e6b60d69b458cc11ae58a016472e1f7ef053adc404935518074747041c163 not found: ID does not exist"
Sep 09 12:53:10.683658 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.683639    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:53:10.688030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:10.688010    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85475bcc65-h5jt4"]
Sep 09 12:53:12.450976 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:12.450943    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" path="/var/lib/kubelet/pods/1b1098bd-4cc5-45d8-9993-399ca33cfbe3/volumes"
Sep 09 12:53:22.371790 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:22.371758    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:53:22.378623 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:22.378596    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:53:22.382168 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:22.382145    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 12:53:35.808811 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:35.808777    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:37.493567 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:37.493528    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:57.563081 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:57.563043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:58.392102 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:58.392068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:53:59.626831 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.626799    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"]
Sep 09 12:53:59.627201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.627080    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" containerName="console"
Sep 09 12:53:59.627201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.627091    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" containerName="console"
Sep 09 12:53:59.627201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.627149    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b1098bd-4cc5-45d8-9993-399ca33cfbe3" containerName="console"
Sep 09 12:53:59.629936 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.629922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.632316 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.632292    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 09 12:53:59.632448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.632292    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-bsxsx\""
Sep 09 12:53:59.632448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.632331    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:53:59.645911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.645892    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"]
Sep 09 12:53:59.687940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.687916    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a825518-e81d-4887-a492-8b61c0f9e0cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.688046 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.687956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snmsk\" (UniqueName: \"kubernetes.io/projected/1a825518-e81d-4887-a492-8b61c0f9e0cb-kube-api-access-snmsk\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.788244 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.788219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a825518-e81d-4887-a492-8b61c0f9e0cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.788330 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.788260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snmsk\" (UniqueName: \"kubernetes.io/projected/1a825518-e81d-4887-a492-8b61c0f9e0cb-kube-api-access-snmsk\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.788622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.788604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1a825518-e81d-4887-a492-8b61c0f9e0cb-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.800341 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.800315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snmsk\" (UniqueName: \"kubernetes.io/projected/1a825518-e81d-4887-a492-8b61c0f9e0cb-kube-api-access-snmsk\") pod \"cert-manager-operator-controller-manager-55c444bf86-kzb8x\" (UID: \"1a825518-e81d-4887-a492-8b61c0f9e0cb\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:53:59.939179 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:53:59.939158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"
Sep 09 12:54:00.215465 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:00.215441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x"]
Sep 09 12:54:00.217520 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:54:00.217475    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a825518_e81d_4887_a492_8b61c0f9e0cb.slice/crio-133659e47d3d41dd064f1f027d2644ccc20b3eed99acf4d6cf4b9d2211d4c9d2 WatchSource:0}: Error finding container 133659e47d3d41dd064f1f027d2644ccc20b3eed99acf4d6cf4b9d2211d4c9d2: Status 404 returned error can't find the container with id 133659e47d3d41dd064f1f027d2644ccc20b3eed99acf4d6cf4b9d2211d4c9d2
Sep 09 12:54:00.220807 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:00.220793    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:54:00.859931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:00.859897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x" event={"ID":"1a825518-e81d-4887-a492-8b61c0f9e0cb","Type":"ContainerStarted","Data":"133659e47d3d41dd064f1f027d2644ccc20b3eed99acf4d6cf4b9d2211d4c9d2"}
Sep 09 12:54:01.176159 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:01.176117    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:01.276025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:01.275906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:03.671116 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:03.671080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:03.875045 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:03.875008    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x" event={"ID":"1a825518-e81d-4887-a492-8b61c0f9e0cb","Type":"ContainerStarted","Data":"5dc0970f460b00e266885e804a406b98d4a994f137ab58e0ac285b6d38e2247e"}
Sep 09 12:54:03.895961 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:03.895915    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-kzb8x" podStartSLOduration=2.280560881 podStartE2EDuration="4.895899307s" totalImagesPullingTime="2.615338426s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:53:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:03.893834708 +0000 UTC m=+341.929082627" watchObservedRunningTime="2026-09-09 12:54:03.895899307 +0000 UTC m=+341.931147214"
Sep 09 12:54:04.430589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.430550    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:04.522980 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.522951    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:04.651338 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.651311    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"]
Sep 09 12:54:04.654742 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.654725    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.658485 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.658464    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:54:04.658741 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.658707    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wclvq\""
Sep 09 12:54:04.658969 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.658953    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:54:04.675701 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.675682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"]
Sep 09 12:54:04.724641 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.724589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.724729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.724642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.724729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.724660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjzb4\" (UniqueName: \"kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.825076 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.825052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.825174 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.825081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjzb4\" (UniqueName: \"kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.825174 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.825116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.825424 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.825406    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.825482 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.825465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.834915 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.834897    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjzb4\" (UniqueName: \"kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:04.963869 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:04.963846    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:05.238097 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:05.238067    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"]
Sep 09 12:54:05.240490 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:54:05.240455    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0d1853_3357_4fa7_b1a5_61d85f61cfad.slice/crio-39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7 WatchSource:0}: Error finding container 39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7: Status 404 returned error can't find the container with id 39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7
Sep 09 12:54:05.639770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:05.639680    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:05.884522 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:05.884475    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb" event={"ID":"4b0d1853-3357-4fa7-b1a5-61d85f61cfad","Type":"ContainerStarted","Data":"39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7"}
Sep 09 12:54:06.056526 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:06.056466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:07.159266 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:07.159225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:07.261331 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:07.261291    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:10.406769 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.406723    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7kxd"]
Sep 09 12:54:10.411739 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.411710    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.414033 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.414005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-kbksv\""
Sep 09 12:54:10.414964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.414945    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 12:54:10.415069 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.414974    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 12:54:10.417791 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.417770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7kxd"]
Sep 09 12:54:10.474652 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.474620    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdpbm\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-kube-api-access-fdpbm\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.474806 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.474702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.575412 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.575380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.575624 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.575444    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdpbm\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-kube-api-access-fdpbm\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.583911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.583850    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.584061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.583952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdpbm\" (UniqueName: \"kubernetes.io/projected/47283061-c2d3-4d51-9e83-8e3530805672-kube-api-access-fdpbm\") pod \"cert-manager-webhook-6486ffd997-f7kxd\" (UID: \"47283061-c2d3-4d51-9e83-8e3530805672\") " pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:10.732596 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:10.732520    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:11.330227 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:11.330199    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-f7kxd"]
Sep 09 12:54:11.332514 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:54:11.332475    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod47283061_c2d3_4d51_9e83_8e3530805672.slice/crio-e8fbb2948835a45211d72b33e02f4423b85522a5e2874a148ce41629799ae074 WatchSource:0}: Error finding container e8fbb2948835a45211d72b33e02f4423b85522a5e2874a148ce41629799ae074: Status 404 returned error can't find the container with id e8fbb2948835a45211d72b33e02f4423b85522a5e2874a148ce41629799ae074
Sep 09 12:54:11.914520 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:11.914470    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerID="8bb361196f9ee139f8121e88292065f31df6e01607002366190cb91dcac12d57" exitCode=0
Sep 09 12:54:11.914961 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:11.914541    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb" event={"ID":"4b0d1853-3357-4fa7-b1a5-61d85f61cfad","Type":"ContainerDied","Data":"8bb361196f9ee139f8121e88292065f31df6e01607002366190cb91dcac12d57"}
Sep 09 12:54:11.916181 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:11.916144    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd" event={"ID":"47283061-c2d3-4d51-9e83-8e3530805672","Type":"ContainerStarted","Data":"e8fbb2948835a45211d72b33e02f4423b85522a5e2874a148ce41629799ae074"}
Sep 09 12:54:14.443083 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.443048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:14.755589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.755484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:14.932434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.932380    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd" event={"ID":"47283061-c2d3-4d51-9e83-8e3530805672","Type":"ContainerStarted","Data":"86937f40c54bb54f5dcb4ecdf3583041341d4a82c8e3633df8595b4c936ec6e0"}
Sep 09 12:54:14.932434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.932433    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:14.934275 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.934251    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerID="0d2f201b6a9353d10684633a91b903ef567e7726b2d3e633b23c370e02dc19d8" exitCode=0
Sep 09 12:54:14.934378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.934281    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb" event={"ID":"4b0d1853-3357-4fa7-b1a5-61d85f61cfad","Type":"ContainerDied","Data":"0d2f201b6a9353d10684633a91b903ef567e7726b2d3e633b23c370e02dc19d8"}
Sep 09 12:54:14.948797 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:14.948747    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd" podStartSLOduration=1.833416416 podStartE2EDuration="4.94872988s" totalImagesPullingTime="3.115313464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:14.947343141 +0000 UTC m=+352.982591049" watchObservedRunningTime="2026-09-09 12:54:14.94872988 +0000 UTC m=+352.983977788"
Sep 09 12:54:20.941597 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:20.941564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-f7kxd"
Sep 09 12:54:22.973633 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:22.973551    2580 generic.go:356] "Generic (PLEG): container finished" podID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerID="0724b8b259c9b52bd5b6bea7c07e24345568be9c6eb53e7eec9e0727c0e94f29" exitCode=0
Sep 09 12:54:22.973989 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:22.973631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb" event={"ID":"4b0d1853-3357-4fa7-b1a5-61d85f61cfad","Type":"ContainerDied","Data":"0724b8b259c9b52bd5b6bea7c07e24345568be9c6eb53e7eec9e0727c0e94f29"}
Sep 09 12:54:24.101978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.101940    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:24.190720 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.190684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle\") pod \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") "
Sep 09 12:54:24.190900 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.190757    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjzb4\" (UniqueName: \"kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4\") pod \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") "
Sep 09 12:54:24.190900 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.190783    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util\") pod \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\" (UID: \"4b0d1853-3357-4fa7-b1a5-61d85f61cfad\") "
Sep 09 12:54:24.191148 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.191119    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle" (OuterVolumeSpecName: "bundle") pod "4b0d1853-3357-4fa7-b1a5-61d85f61cfad" (UID: "4b0d1853-3357-4fa7-b1a5-61d85f61cfad"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:54:24.192945 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.192925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4" (OuterVolumeSpecName: "kube-api-access-hjzb4") pod "4b0d1853-3357-4fa7-b1a5-61d85f61cfad" (UID: "4b0d1853-3357-4fa7-b1a5-61d85f61cfad"). InnerVolumeSpecName "kube-api-access-hjzb4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:54:24.194980 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.194961    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util" (OuterVolumeSpecName: "util") pod "4b0d1853-3357-4fa7-b1a5-61d85f61cfad" (UID: "4b0d1853-3357-4fa7-b1a5-61d85f61cfad"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:54:24.291535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.291413    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjzb4\" (UniqueName: \"kubernetes.io/projected/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-kube-api-access-hjzb4\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:54:24.291535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.291457    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:54:24.291535 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.291467    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4b0d1853-3357-4fa7-b1a5-61d85f61cfad-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:54:24.544291 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544217    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8zql"]
Sep 09 12:54:24.544527 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544515    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="pull"
Sep 09 12:54:24.544577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544528    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="pull"
Sep 09 12:54:24.544577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544540    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="util"
Sep 09 12:54:24.544577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544545    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="util"
Sep 09 12:54:24.544577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544554    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="extract"
Sep 09 12:54:24.544577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544559    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="extract"
Sep 09 12:54:24.544749 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.544610    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0d1853-3357-4fa7-b1a5-61d85f61cfad" containerName="extract"
Sep 09 12:54:24.547445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.547427    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.549538 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.549519    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-9k82r\""
Sep 09 12:54:24.557796 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.557774    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8zql"]
Sep 09 12:54:24.593688 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.593666    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tltl8\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-kube-api-access-tltl8\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.593788 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.593697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.694662 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.694624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tltl8\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-kube-api-access-tltl8\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.694662 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.694664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.702808 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.702773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.702919 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.702842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tltl8\" (UniqueName: \"kubernetes.io/projected/1d7f5cf8-f26f-498e-a6ad-621d8868b9fc-kube-api-access-tltl8\") pod \"cert-manager-55b4d954d5-c8zql\" (UID: \"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc\") " pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.856139 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.856063    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c8zql"
Sep 09 12:54:24.984057 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.984029    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb"
Sep 09 12:54:24.984335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.984029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fp47hb" event={"ID":"4b0d1853-3357-4fa7-b1a5-61d85f61cfad","Type":"ContainerDied","Data":"39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7"}
Sep 09 12:54:24.984335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:24.984149    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="39a3275e35e49554ddad8072a2fea75cfb0a7aadc32889515243e538fa4959d7"
Sep 09 12:54:25.123716 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:25.123480    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8zql"]
Sep 09 12:54:25.125891 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:54:25.125866    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1d7f5cf8_f26f_498e_a6ad_621d8868b9fc.slice/crio-03c553209b76a558bee900f7f5575b7cdb3caa10c0666be51e493c0c7760d7af WatchSource:0}: Error finding container 03c553209b76a558bee900f7f5575b7cdb3caa10c0666be51e493c0c7760d7af: Status 404 returned error can't find the container with id 03c553209b76a558bee900f7f5575b7cdb3caa10c0666be51e493c0c7760d7af
Sep 09 12:54:25.990042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:25.990007    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c8zql" event={"ID":"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc","Type":"ContainerStarted","Data":"de72bce7c35219fd0177935a80e6cc2213ec01006b39feaf089eecc6e6a1b8b9"}
Sep 09 12:54:25.990042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:25.990047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c8zql" event={"ID":"1d7f5cf8-f26f-498e-a6ad-621d8868b9fc","Type":"ContainerStarted","Data":"03c553209b76a558bee900f7f5575b7cdb3caa10c0666be51e493c0c7760d7af"}
Sep 09 12:54:26.004965 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:26.004913    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c8zql" podStartSLOduration=2.004898778 podStartE2EDuration="2.004898778s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:26.003660348 +0000 UTC m=+364.038908256" watchObservedRunningTime="2026-09-09 12:54:26.004898778 +0000 UTC m=+364.040146682"
Sep 09 12:54:29.178885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:29.178848    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:29.274855 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:29.274829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:30.867018 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.866982    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"]
Sep 09 12:54:30.918141 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.918111    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"]
Sep 09 12:54:30.918276 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.918210    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:30.922425 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.922399    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 09 12:54:30.922668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.922651    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 09 12:54:30.924061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:30.924043    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2pr8f\""
Sep 09 12:54:31.041110 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.041079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-th8jm\" (UniqueName: \"kubernetes.io/projected/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-kube-api-access-th8jm\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.041258 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.041127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.141893 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.141872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-th8jm\" (UniqueName: \"kubernetes.io/projected/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-kube-api-access-th8jm\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.142012 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.141917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.142226 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.142210    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-tmp\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.149494 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.149467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-th8jm\" (UniqueName: \"kubernetes.io/projected/207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15-kube-api-access-th8jm\") pod \"openshift-lws-operator-fd8ccff4c-5zkqj\" (UID: \"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.227023 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.227000    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"
Sep 09 12:54:31.515074 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:31.515050    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj"]
Sep 09 12:54:31.517315 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:54:31.517284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod207dfaa1_41ee_4e9a_97c8_7c4ef0b41c15.slice/crio-950b7d67fc850d99dbd64989599d720aa74da3212f44d5ecee8d0124644a1276 WatchSource:0}: Error finding container 950b7d67fc850d99dbd64989599d720aa74da3212f44d5ecee8d0124644a1276: Status 404 returned error can't find the container with id 950b7d67fc850d99dbd64989599d720aa74da3212f44d5ecee8d0124644a1276
Sep 09 12:54:32.016486 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:32.016452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj" event={"ID":"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15","Type":"ContainerStarted","Data":"950b7d67fc850d99dbd64989599d720aa74da3212f44d5ecee8d0124644a1276"}
Sep 09 12:54:45.074362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:45.074326    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj" event={"ID":"207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15","Type":"ContainerStarted","Data":"4915fabc5c685adcd1d09342f56bd0195466accd0667c97ff30258d7701b89bd"}
Sep 09 12:54:45.090169 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:45.090113    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-5zkqj" podStartSLOduration=2.575802781 podStartE2EDuration="15.09009932s" totalImagesPullingTime="12.514296539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:54:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:54:45.088897573 +0000 UTC m=+383.124145458" watchObservedRunningTime="2026-09-09 12:54:45.09009932 +0000 UTC m=+383.125347226"
Sep 09 12:54:45.883064 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:45.883029    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:45.988700 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:45.988666    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:47.853920 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:47.853882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:52.143641 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:52.143603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:54:52.237281 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:54:52.237251    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:01.288038 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.288008    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64d54f989d-4wmhs"]
Sep 09 12:55:01.290203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.290185    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.316901 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.316872    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d54f989d-4wmhs"]
Sep 09 12:55:01.385776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29cds\" (UniqueName: \"kubernetes.io/projected/825510a8-2d4b-4adc-a7fd-2f5f4c501683-kube-api-access-29cds\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.385776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385748    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.385776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385763    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-oauth-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.385776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385778    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-trusted-ca-bundle\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.386061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385839    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-oauth-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.386061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-service-ca\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.386061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.385958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.486985 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.486947    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-service-ca\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.486998    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487023    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29cds\" (UniqueName: \"kubernetes.io/projected/825510a8-2d4b-4adc-a7fd-2f5f4c501683-kube-api-access-29cds\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487065    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-oauth-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-trusted-ca-bundle\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487138 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-oauth-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487828 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487801    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-service-ca\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487828 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.487963 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.487814    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-oauth-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.488072 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.488053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/825510a8-2d4b-4adc-a7fd-2f5f4c501683-trusted-ca-bundle\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.489616 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.489587    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-oauth-config\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.489616 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.489604    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/825510a8-2d4b-4adc-a7fd-2f5f4c501683-console-serving-cert\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.495347 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.495328    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29cds\" (UniqueName: \"kubernetes.io/projected/825510a8-2d4b-4adc-a7fd-2f5f4c501683-kube-api-access-29cds\") pod \"console-64d54f989d-4wmhs\" (UID: \"825510a8-2d4b-4adc-a7fd-2f5f4c501683\") " pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.600221 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.600192    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:01.870540 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:01.870452    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64d54f989d-4wmhs"]
Sep 09 12:55:01.876548 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:01.876517    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod825510a8_2d4b_4adc_a7fd_2f5f4c501683.slice/crio-9c3fc4b60e0eb9161a2947fa7c557bed1409551795791759fa0a62c0149baaa5 WatchSource:0}: Error finding container 9c3fc4b60e0eb9161a2947fa7c557bed1409551795791759fa0a62c0149baaa5: Status 404 returned error can't find the container with id 9c3fc4b60e0eb9161a2947fa7c557bed1409551795791759fa0a62c0149baaa5
Sep 09 12:55:02.148018 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.147982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d54f989d-4wmhs" event={"ID":"825510a8-2d4b-4adc-a7fd-2f5f4c501683","Type":"ContainerStarted","Data":"4a58e6c4acb717734514497129da51e7ee0f603ec774f876661a0ed7b8dc4e91"}
Sep 09 12:55:02.148179 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.148033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64d54f989d-4wmhs" event={"ID":"825510a8-2d4b-4adc-a7fd-2f5f4c501683","Type":"ContainerStarted","Data":"9c3fc4b60e0eb9161a2947fa7c557bed1409551795791759fa0a62c0149baaa5"}
Sep 09 12:55:02.182327 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.182237    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64d54f989d-4wmhs" podStartSLOduration=1.182223001 podStartE2EDuration="1.182223001s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:02.179721524 +0000 UTC m=+400.214969432" watchObservedRunningTime="2026-09-09 12:55:02.182223001 +0000 UTC m=+400.217470907"
Sep 09 12:55:02.259427 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.259395    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"]
Sep 09 12:55:02.261576 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.261559    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.263887 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.263860    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:55:02.264025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.263946    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:55:02.264025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.263946    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wclvq\""
Sep 09 12:55:02.271307 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.271286    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"]
Sep 09 12:55:02.294155 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.294129    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.294468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.294179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnhtk\" (UniqueName: \"kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.294468 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.294252    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.395568 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.395541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.395691 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.395593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnhtk\" (UniqueName: \"kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.395691 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.395615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.395926 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.395904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.395980 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.395912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.404320 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.404302    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnhtk\" (UniqueName: \"kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.570738 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.570658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:02.592746 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.592718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:02.839588 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:02.839552    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"]
Sep 09 12:55:02.841643 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:02.841619    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6e1e9dd_ac20_4b35_9714_11e45e85bb08.slice/crio-adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c WatchSource:0}: Error finding container adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c: Status 404 returned error can't find the container with id adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c
Sep 09 12:55:03.028899 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.028865    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:03.126616 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.126519    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:03.154271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.154239    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerID="418e389276fc4eb841db6091fa85807f72476e02a840576ec75c19e84bf2031d" exitCode=0
Sep 09 12:55:03.154438 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.154325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm" event={"ID":"d6e1e9dd-ac20-4b35-9714-11e45e85bb08","Type":"ContainerDied","Data":"418e389276fc4eb841db6091fa85807f72476e02a840576ec75c19e84bf2031d"}
Sep 09 12:55:03.154438 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.154373    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm" event={"ID":"d6e1e9dd-ac20-4b35-9714-11e45e85bb08","Type":"ContainerStarted","Data":"adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c"}
Sep 09 12:55:03.633166 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.633132    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:03.987621 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.987585    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"]
Sep 09 12:55:03.990202 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.990181    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:03.992554 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.992532    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-vtl8s\""
Sep 09 12:55:03.992713 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.992692    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 12:55:03.992805 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.992787    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 12:55:03.993162 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.993148    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 12:55:03.993311 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:03.993294    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 12:55:04.008236 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.008213    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.008335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.008258    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.008335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.008277    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.008435 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.008327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.008471 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.008429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcbdp\" (UniqueName: \"kubernetes.io/projected/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-kube-api-access-dcbdp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.010785 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.010764    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"]
Sep 09 12:55:04.108817 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.108758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcbdp\" (UniqueName: \"kubernetes.io/projected/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-kube-api-access-dcbdp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.108817 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.108794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.108995 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.108843    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.108995 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.108862    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.108995 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.108878    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.109270 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.109245    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-manifests\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.111060 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.111038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-tmp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.111271 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.111255    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-apiservice-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.111340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.111325    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-webhook-cert\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.116480 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.116460    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcbdp\" (UniqueName: \"kubernetes.io/projected/ec32ce95-8011-4a26-b5c1-3b9e3138eeb5-kube-api-access-dcbdp\") pod \"opendatahub-operator-controller-manager-647774d94-dxwnv\" (UID: \"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5\") " pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.152523 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.152473    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:04.160952 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.160933    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerID="5af3f7a191fd339b73b7536890b6445c39b9c1fdf2fabbe96a990eddd81f8f6a" exitCode=0
Sep 09 12:55:04.161030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.160971    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm" event={"ID":"d6e1e9dd-ac20-4b35-9714-11e45e85bb08","Type":"ContainerDied","Data":"5af3f7a191fd339b73b7536890b6445c39b9c1fdf2fabbe96a990eddd81f8f6a"}
Sep 09 12:55:04.270010 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.269962    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:04.300432 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.300403    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:04.367321 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.367261    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:04.476092 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.476060    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:04.580786 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.580758    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:04.582077 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:04.582055    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"]
Sep 09 12:55:04.584679 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:04.584651    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec32ce95_8011_4a26_b5c1_3b9e3138eeb5.slice/crio-5d686767d756d79a76d0fd83856a34f26a1a0461596d00fed10c09e71c2df1c4 WatchSource:0}: Error finding container 5d686767d756d79a76d0fd83856a34f26a1a0461596d00fed10c09e71c2df1c4: Status 404 returned error can't find the container with id 5d686767d756d79a76d0fd83856a34f26a1a0461596d00fed10c09e71c2df1c4
Sep 09 12:55:05.170141 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:05.170084    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerID="2a75986e8412ddb8741d3dc4be21e3b7465fde16f007127f193d6022c17b3bc8" exitCode=0
Sep 09 12:55:05.170623 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:05.170172    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm" event={"ID":"d6e1e9dd-ac20-4b35-9714-11e45e85bb08","Type":"ContainerDied","Data":"2a75986e8412ddb8741d3dc4be21e3b7465fde16f007127f193d6022c17b3bc8"}
Sep 09 12:55:05.172077 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:05.172051    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv" event={"ID":"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5","Type":"ContainerStarted","Data":"5d686767d756d79a76d0fd83856a34f26a1a0461596d00fed10c09e71c2df1c4"}
Sep 09 12:55:07.493414 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.493389    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:07.538526 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.538396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util\") pod \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") "
Sep 09 12:55:07.538526 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.538466    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnhtk\" (UniqueName: \"kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk\") pod \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") "
Sep 09 12:55:07.538526 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.538521    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle\") pod \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\" (UID: \"d6e1e9dd-ac20-4b35-9714-11e45e85bb08\") "
Sep 09 12:55:07.539803 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.539747    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle" (OuterVolumeSpecName: "bundle") pod "d6e1e9dd-ac20-4b35-9714-11e45e85bb08" (UID: "d6e1e9dd-ac20-4b35-9714-11e45e85bb08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:07.541045 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.541018    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk" (OuterVolumeSpecName: "kube-api-access-tnhtk") pod "d6e1e9dd-ac20-4b35-9714-11e45e85bb08" (UID: "d6e1e9dd-ac20-4b35-9714-11e45e85bb08"). InnerVolumeSpecName "kube-api-access-tnhtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:07.544845 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.544822    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util" (OuterVolumeSpecName: "util") pod "d6e1e9dd-ac20-4b35-9714-11e45e85bb08" (UID: "d6e1e9dd-ac20-4b35-9714-11e45e85bb08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:07.640025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.640004    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:07.640025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.640025    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnhtk\" (UniqueName: \"kubernetes.io/projected/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-kube-api-access-tnhtk\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:07.640145 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:07.640034    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d6e1e9dd-ac20-4b35-9714-11e45e85bb08-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:08.190042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:08.190015    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm"
Sep 09 12:55:08.190218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:08.190044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b02g2rm" event={"ID":"d6e1e9dd-ac20-4b35-9714-11e45e85bb08","Type":"ContainerDied","Data":"adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c"}
Sep 09 12:55:08.190218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:08.190115    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="adbc2e62e6598de7d1a6eac49c9f3ddc7d33f482880fc23f6ad37d1c8f27d20c"
Sep 09 12:55:08.191756 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:08.191730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv" event={"ID":"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5","Type":"ContainerStarted","Data":"7178a559b9725a286fde26c111adf736d56b27b5f69d5fdc48b73be36b9b202f"}
Sep 09 12:55:09.197607 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:09.197566    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec32ce95-8011-4a26-b5c1-3b9e3138eeb5" containerID="7178a559b9725a286fde26c111adf736d56b27b5f69d5fdc48b73be36b9b202f" exitCode=0
Sep 09 12:55:09.197996 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:09.197646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv" event={"ID":"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5","Type":"ContainerDied","Data":"7178a559b9725a286fde26c111adf736d56b27b5f69d5fdc48b73be36b9b202f"}
Sep 09 12:55:10.204107 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:10.204071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv" event={"ID":"ec32ce95-8011-4a26-b5c1-3b9e3138eeb5","Type":"ContainerStarted","Data":"a6ff720034be9fb6b05db7f9b8df99206c378f323488680661f917335aaa94d0"}
Sep 09 12:55:10.204518 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:10.204151    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:10.227555 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:10.227490    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv" podStartSLOduration=2.901813836 podStartE2EDuration="7.227477139s" totalImagesPullingTime="3.309153489s" totalInitContainerRuntime="1.016509814s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:10.224945092 +0000 UTC m=+408.260193000" watchObservedRunningTime="2026-09-09 12:55:10.227477139 +0000 UTC m=+408.262725046"
Sep 09 12:55:11.600594 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:11.600559    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:11.600958 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:11.600615    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:11.605072 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:11.605053    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:12.221384 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:12.221354    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64d54f989d-4wmhs"
Sep 09 12:55:12.268067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:12.268036    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:55:12.372047 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:12.372013    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:12.474861 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:12.474772    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:12.572751 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:12.572717    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:21.215205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:21.215175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-647774d94-dxwnv"
Sep 09 12:55:24.120108 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.120067    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:24.249956 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.249924    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:24.411410 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.411375    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:24.415747 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.415724    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-768599446d-wl78j"]
Sep 09 12:55:24.416039 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416027    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="pull"
Sep 09 12:55:24.416096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416041    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="pull"
Sep 09 12:55:24.416096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416055    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="extract"
Sep 09 12:55:24.416096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416061    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="extract"
Sep 09 12:55:24.416096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416075    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="util"
Sep 09 12:55:24.416096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416082    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="util"
Sep 09 12:55:24.416253 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.416136    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6e1e9dd-ac20-4b35-9714-11e45e85bb08" containerName="extract"
Sep 09 12:55:24.422109 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.422092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.426964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.426946    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 12:55:24.427065 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.426948    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 12:55:24.435729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.435705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-768599446d-wl78j"]
Sep 09 12:55:24.463761 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.463694    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6934b57-ea63-4e20-8245-2a833e114f76-tls-certs\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.464072 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.464044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8j5hg\" (UniqueName: \"kubernetes.io/projected/a6934b57-ea63-4e20-8245-2a833e114f76-kube-api-access-8j5hg\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.464168 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.464121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6934b57-ea63-4e20-8245-2a833e114f76-tmp\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.515604 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.515567    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:24.565011 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.564972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8j5hg\" (UniqueName: \"kubernetes.io/projected/a6934b57-ea63-4e20-8245-2a833e114f76-kube-api-access-8j5hg\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.565185 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.565027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6934b57-ea63-4e20-8245-2a833e114f76-tmp\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.565185 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.565072    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6934b57-ea63-4e20-8245-2a833e114f76-tls-certs\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.567546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.567497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a6934b57-ea63-4e20-8245-2a833e114f76-tmp\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.567774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.567753    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a6934b57-ea63-4e20-8245-2a833e114f76-tls-certs\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.586355 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.586327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8j5hg\" (UniqueName: \"kubernetes.io/projected/a6934b57-ea63-4e20-8245-2a833e114f76-kube-api-access-8j5hg\") pod \"kube-auth-proxy-768599446d-wl78j\" (UID: \"a6934b57-ea63-4e20-8245-2a833e114f76\") " pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:24.731933 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:24.731873    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j"
Sep 09 12:55:25.002123 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.002099    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-768599446d-wl78j"]
Sep 09 12:55:25.004328 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:25.004299    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6934b57_ea63_4e20_8245_2a833e114f76.slice/crio-d0cdae31c2b1c7d06cc3d9a65057324e25b58e21100613941fad91f9004573ff WatchSource:0}: Error finding container d0cdae31c2b1c7d06cc3d9a65057324e25b58e21100613941fad91f9004573ff: Status 404 returned error can't find the container with id d0cdae31c2b1c7d06cc3d9a65057324e25b58e21100613941fad91f9004573ff
Sep 09 12:55:25.287908 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.287834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j" event={"ID":"a6934b57-ea63-4e20-8245-2a833e114f76","Type":"ContainerStarted","Data":"d0cdae31c2b1c7d06cc3d9a65057324e25b58e21100613941fad91f9004573ff"}
Sep 09 12:55:25.727435 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.727398    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:25.731783 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.731744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:25.732863 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.732847    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.735180 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.735156    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 12:55:25.735180 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.735169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 12:55:25.735343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.735192    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 12:55:25.735343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.735200    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-rmv6s\""
Sep 09 12:55:25.735343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.735211    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 12:55:25.746975 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.746954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:25.775345 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775374    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775554 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775467    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775554 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbxvd\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775554 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775548    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.775665 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.775579    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.795815 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.795789    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:25.876566 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876540    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876667 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876592    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876667 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876619    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876667 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876834 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876812    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876894 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876851    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbxvd\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.876894 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.876882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.877294 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.877274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.878931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.878910    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.879055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.879038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.879373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.879356    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.879483 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.879462    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.884780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.884757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbxvd\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:25.884857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:25.884765    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-strrr\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:26.042545 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:26.042471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:26.312885 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:26.312859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:26.315234 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:26.315204    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe379105_d981_4d02_ba1e_de25ebe72463.slice/crio-be350b2e7cb4eae23cfda04c8937174e5a991b3648483d6d19c4e8af5bb4a2c2 WatchSource:0}: Error finding container be350b2e7cb4eae23cfda04c8937174e5a991b3648483d6d19c4e8af5bb4a2c2: Status 404 returned error can't find the container with id be350b2e7cb4eae23cfda04c8937174e5a991b3648483d6d19c4e8af5bb4a2c2
Sep 09 12:55:26.623150 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:26.623074    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:27.299807 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:27.299766    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" event={"ID":"be379105-d981-4d02-ba1e-de25ebe72463","Type":"ContainerStarted","Data":"be350b2e7cb4eae23cfda04c8937174e5a991b3648483d6d19c4e8af5bb4a2c2"}
Sep 09 12:55:30.301831 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:30.301786    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 12:55:30.302114 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:30.301870    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 12:55:30.988567 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:30.988531    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:31.046446 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.046407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:31.190305 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.190270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:31.290181 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.290103    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:31.323473 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.323436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j" event={"ID":"a6934b57-ea63-4e20-8245-2a833e114f76","Type":"ContainerStarted","Data":"efb95ccb1c7aae561c0362290d0f60ef8a9b31ef0ccc598de8af009d47a49091"}
Sep 09 12:55:31.324966 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.324947    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" event={"ID":"be379105-d981-4d02-ba1e-de25ebe72463","Type":"ContainerStarted","Data":"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"}
Sep 09 12:55:31.325064 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.324988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:31.326691 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.326664    2580 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-strrr container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 12:55:31.326796 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.326714    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" podUID="be379105-d981-4d02-ba1e-de25ebe72463" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 12:55:31.342018 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.341976    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-768599446d-wl78j" podStartSLOduration=1.9981513290000001 podStartE2EDuration="7.341955756s" totalImagesPullingTime="5.343804427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:31.339932405 +0000 UTC m=+429.375180315" watchObservedRunningTime="2026-09-09 12:55:31.341955756 +0000 UTC m=+429.377203671"
Sep 09 12:55:31.361355 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.361312    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" podStartSLOduration=2.377014938 podStartE2EDuration="6.361296316s" totalImagesPullingTime="3.984281378s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:31.359462994 +0000 UTC m=+429.394710912" watchObservedRunningTime="2026-09-09 12:55:31.361296316 +0000 UTC m=+429.396544222"
Sep 09 12:55:31.770611 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.770574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:31.987029 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:31.986992    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:32.122534 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:32.122428    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:32.330812 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:32.330785    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:35.195374 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:35.195321    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:35.298148 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:35.298104    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:35.406149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:35.406116    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:35.571591 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:35.571491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:35.886398 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:35.886313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:37.295929 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.295886    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b9fd88b7f-6zpzn" podUID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" containerName="console" containerID="cri-o://64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1" gracePeriod=15
Sep 09 12:55:37.546931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.546868    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:55:37.680382 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680336    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680401    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9vn6\" (UniqueName: \"kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680434    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680551    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680800 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680800 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680732    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:37.680911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config\") pod \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\" (UID: \"e3c200d1-6dd4-4d0e-813c-3fa0f567667a\") "
Sep 09 12:55:37.680963 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680898    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca" (OuterVolumeSpecName: "service-ca") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:37.681025 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.680983    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config" (OuterVolumeSpecName: "console-config") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:37.681069 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.681042    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:37.681154 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.681138    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.681199 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.681157    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.681199 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.681167    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-oauth-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.681199 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.681175    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-trusted-ca-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.682676 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.682653    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6" (OuterVolumeSpecName: "kube-api-access-d9vn6") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "kube-api-access-d9vn6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:37.682993 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.682979    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:37.683054 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.682998    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e3c200d1-6dd4-4d0e-813c-3fa0f567667a" (UID: "e3c200d1-6dd4-4d0e-813c-3fa0f567667a"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:37.782292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.782259    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-oauth-config\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.782292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.782289    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9vn6\" (UniqueName: \"kubernetes.io/projected/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-kube-api-access-d9vn6\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:37.782292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:37.782301    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e3c200d1-6dd4-4d0e-813c-3fa0f567667a-console-serving-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:38.366095 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.366057    2580 generic.go:356] "Generic (PLEG): container finished" podID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" containerID="64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1" exitCode=0
Sep 09 12:55:38.366589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.366118    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b9fd88b7f-6zpzn"
Sep 09 12:55:38.366589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.366150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd88b7f-6zpzn" event={"ID":"e3c200d1-6dd4-4d0e-813c-3fa0f567667a","Type":"ContainerDied","Data":"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"}
Sep 09 12:55:38.366589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.366239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b9fd88b7f-6zpzn" event={"ID":"e3c200d1-6dd4-4d0e-813c-3fa0f567667a","Type":"ContainerDied","Data":"e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075"}
Sep 09 12:55:38.366589 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.366265    2580 scope.go:117] "RemoveContainer" containerID="64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"
Sep 09 12:55:38.376648 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.376621    2580 scope.go:117] "RemoveContainer" containerID="64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"
Sep 09 12:55:38.377070 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:55:38.377050    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1\": container with ID starting with 64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1 not found: ID does not exist" containerID="64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"
Sep 09 12:55:38.377135 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.377078    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1"} err="failed to get container status \"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1\": rpc error: code = NotFound desc = could not find container \"64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1\": container with ID starting with 64cbf5e251e3bd82d168f60be48300fb630f63d1ea2ad59a5f05ec20d05a03e1 not found: ID does not exist"
Sep 09 12:55:38.390992 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.390956    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:55:38.399071 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.399043    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b9fd88b7f-6zpzn"]
Sep 09 12:55:38.404137 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:55:38.404111    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c200d1_6dd4_4d0e_813c_3fa0f567667a.slice\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3c200d1_6dd4_4d0e_813c_3fa0f567667a.slice/crio-e48161e188c252c558a723a84a774729941d416aa31d84f7e9b216d3329ef075\": RecentStats: unable to find data in memory cache]"
Sep 09 12:55:38.452113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:38.452086    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" path="/var/lib/kubelet/pods/e3c200d1-6dd4-4d0e-813c-3fa0f567667a/volumes"
Sep 09 12:55:44.663157 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.663110    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:44.720091 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.720048    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"]
Sep 09 12:55:44.720564 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.720547    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" containerName="console"
Sep 09 12:55:44.720564 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.720564    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" containerName="console"
Sep 09 12:55:44.720706 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.720631    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e3c200d1-6dd4-4d0e-813c-3fa0f567667a" containerName="console"
Sep 09 12:55:44.731304 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.731281    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"]
Sep 09 12:55:44.731419 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.731395    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.733761 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.733736    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-p2p2w\""
Sep 09 12:55:44.734015 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.733997    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 12:55:44.734755 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.734709    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.735080 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.735059    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"]
Sep 09 12:55:44.737131 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.737106    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 09 12:55:44.737288 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.737271    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-sw9mm\""
Sep 09 12:55:44.748041 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.748020    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"]
Sep 09 12:55:44.774155 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.774124    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:44.840193 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.840119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dns2\" (UniqueName: \"kubernetes.io/projected/8bf8f3b2-e153-424b-9805-35ab0c027117-kube-api-access-5dns2\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.840193 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.840160    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d60c981-2889-4148-a482-9588c452044e-cert\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.840402 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.840238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-595gn\" (UniqueName: \"kubernetes.io/projected/0d60c981-2889-4148-a482-9588c452044e-kube-api-access-595gn\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.840402 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.840273    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bf8f3b2-e153-424b-9805-35ab0c027117-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.856796 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.856771    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:44.941426 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.941346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dns2\" (UniqueName: \"kubernetes.io/projected/8bf8f3b2-e153-424b-9805-35ab0c027117-kube-api-access-5dns2\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.941426 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.941393    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d60c981-2889-4148-a482-9588c452044e-cert\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.941426 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.941421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-595gn\" (UniqueName: \"kubernetes.io/projected/0d60c981-2889-4148-a482-9588c452044e-kube-api-access-595gn\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.941731 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.941436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bf8f3b2-e153-424b-9805-35ab0c027117-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.943874 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.943852    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8bf8f3b2-e153-424b-9805-35ab0c027117-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.944026 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.944003    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0d60c981-2889-4148-a482-9588c452044e-cert\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.958240 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.958205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-595gn\" (UniqueName: \"kubernetes.io/projected/0d60c981-2889-4148-a482-9588c452044e-kube-api-access-595gn\") pod \"kserve-controller-manager-d6f4b65b8-7gzcx\" (UID: \"0d60c981-2889-4148-a482-9588c452044e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:44.960339 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.960318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dns2\" (UniqueName: \"kubernetes.io/projected/8bf8f3b2-e153-424b-9805-35ab0c027117-kube-api-access-5dns2\") pod \"llmisvc-controller-manager-777c6bd7d5-9ppks\" (UID: \"8bf8f3b2-e153-424b-9805-35ab0c027117\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:44.968949 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:44.968920    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:45.045344 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.045284    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:45.051999 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.051971    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:45.329257 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.329231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"]
Sep 09 12:55:45.331629 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:45.331602    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d60c981_2889_4148_a482_9588c452044e.slice/crio-6eda8dcf1ae616fefa57783f7127b9220b8a569b68a7f9ab6731654aadb82f7c WatchSource:0}: Error finding container 6eda8dcf1ae616fefa57783f7127b9220b8a569b68a7f9ab6731654aadb82f7c: Status 404 returned error can't find the container with id 6eda8dcf1ae616fefa57783f7127b9220b8a569b68a7f9ab6731654aadb82f7c
Sep 09 12:55:45.334919 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.334899    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"]
Sep 09 12:55:45.336630 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:55:45.336598    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8bf8f3b2_e153_424b_9805_35ab0c027117.slice/crio-fff72eba4625c5d65b87d140e5f61a95e0718dda9add4d3c5aeb2eb1acf24d84 WatchSource:0}: Error finding container fff72eba4625c5d65b87d140e5f61a95e0718dda9add4d3c5aeb2eb1acf24d84: Status 404 returned error can't find the container with id fff72eba4625c5d65b87d140e5f61a95e0718dda9add4d3c5aeb2eb1acf24d84
Sep 09 12:55:45.408173 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.408143    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx" event={"ID":"0d60c981-2889-4148-a482-9588c452044e","Type":"ContainerStarted","Data":"6eda8dcf1ae616fefa57783f7127b9220b8a569b68a7f9ab6731654aadb82f7c"}
Sep 09 12:55:45.409272 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:45.409250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks" event={"ID":"8bf8f3b2-e153-424b-9805-35ab0c027117","Type":"ContainerStarted","Data":"fff72eba4625c5d65b87d140e5f61a95e0718dda9add4d3c5aeb2eb1acf24d84"}
Sep 09 12:55:46.004915 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:46.004885    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:46.103896 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:46.103860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:55:49.244278 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.244241    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:49.244776 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.244491    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" podUID="be379105-d981-4d02-ba1e-de25ebe72463" containerName="discovery" containerID="cri-o://4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3" gracePeriod=30
Sep 09 12:55:49.624841 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.624818    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:49.782140 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782061    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782140 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782110    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782140 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782169    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782214    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782237    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782407 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782295    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbxvd\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd\") pod \"be379105-d981-4d02-ba1e-de25ebe72463\" (UID: \"be379105-d981-4d02-ba1e-de25ebe72463\") "
Sep 09 12:55:49.782663 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.782634    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:55:49.784693 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.784657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts" (OuterVolumeSpecName: "cacerts") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:49.784868 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.784847    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs" (OuterVolumeSpecName: "local-certs") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:55:49.784938 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.784907    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:49.785001 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.784938    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd" (OuterVolumeSpecName: "kube-api-access-mbxvd") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "kube-api-access-mbxvd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:49.785054 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.785035    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:55:49.785210 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.785191    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token" (OuterVolumeSpecName: "istio-token") pod "be379105-d981-4d02-ba1e-de25ebe72463" (UID: "be379105-d981-4d02-ba1e-de25ebe72463"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:55:49.883540 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883494    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-dns-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883540 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883537    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-istio-token\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883547    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbxvd\" (UniqueName: \"kubernetes.io/projected/be379105-d981-4d02-ba1e-de25ebe72463-kube-api-access-mbxvd\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883557    2580 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-cacerts\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883566    2580 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/be379105-d981-4d02-ba1e-de25ebe72463-istio-csr-ca-configmap\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883575    2580 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/be379105-d981-4d02-ba1e-de25ebe72463-local-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:49.883692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:49.883583    2580 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/be379105-d981-4d02-ba1e-de25ebe72463-istio-kubeconfig\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:55:50.439821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.439785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks" event={"ID":"8bf8f3b2-e153-424b-9805-35ab0c027117","Type":"ContainerStarted","Data":"3841dd9d54c1ea08cc69ce9b01a76b4dc2078c149a24d7f38146caf4029c704e"}
Sep 09 12:55:50.440266 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.439889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:55:50.441247 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.441227    2580 generic.go:356] "Generic (PLEG): container finished" podID="be379105-d981-4d02-ba1e-de25ebe72463" containerID="4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3" exitCode=0
Sep 09 12:55:50.441297 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.441262    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"
Sep 09 12:55:50.441297 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.441290    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" event={"ID":"be379105-d981-4d02-ba1e-de25ebe72463","Type":"ContainerDied","Data":"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"}
Sep 09 12:55:50.441368 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.441323    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr" event={"ID":"be379105-d981-4d02-ba1e-de25ebe72463","Type":"ContainerDied","Data":"be350b2e7cb4eae23cfda04c8937174e5a991b3648483d6d19c4e8af5bb4a2c2"}
Sep 09 12:55:50.441368 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.441342    2580 scope.go:117] "RemoveContainer" containerID="4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"
Sep 09 12:55:50.443021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.442991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx" event={"ID":"0d60c981-2889-4148-a482-9588c452044e","Type":"ContainerStarted","Data":"a17ba0419bf5123b0af911a89611dae4d89354dca5c81a61767e8d21119fd1b9"}
Sep 09 12:55:50.443132 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.443047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:55:50.450614 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.450598    2580 scope.go:117] "RemoveContainer" containerID="4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"
Sep 09 12:55:50.450838 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:55:50.450820    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3\": container with ID starting with 4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3 not found: ID does not exist" containerID="4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"
Sep 09 12:55:50.450964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.450845    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3"} err="failed to get container status \"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3\": rpc error: code = NotFound desc = could not find container \"4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3\": container with ID starting with 4fd91aa609148d7f6b969ee327b25567c31834ae6a1d68b5e61989a23a6afee3 not found: ID does not exist"
Sep 09 12:55:50.457216 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.457177    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks" podStartSLOduration=2.273527264 podStartE2EDuration="6.457165919s" totalImagesPullingTime="4.183638655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:50.455036471 +0000 UTC m=+448.490284377" watchObservedRunningTime="2026-09-09 12:55:50.457165919 +0000 UTC m=+448.492413826"
Sep 09 12:55:50.472397 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.472358    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx" podStartSLOduration=2.934090006 podStartE2EDuration="6.472341718s" totalImagesPullingTime="3.538251712s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:55:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:55:50.469361086 +0000 UTC m=+448.504608994" watchObservedRunningTime="2026-09-09 12:55:50.472341718 +0000 UTC m=+448.507589626"
Sep 09 12:55:50.483814 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.483792    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:50.489587 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:50.489570    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-strrr"]
Sep 09 12:55:52.452224 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:52.452188    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be379105-d981-4d02-ba1e-de25ebe72463" path="/var/lib/kubelet/pods/be379105-d981-4d02-ba1e-de25ebe72463/volumes"
Sep 09 12:55:57.464757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:55:57.464721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:01.472192 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:01.472156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:01.525451 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:01.525418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:21.457056 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:21.457022    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-9ppks"
Sep 09 12:56:21.460211 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:21.460187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-7gzcx"
Sep 09 12:56:36.538328 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:36.538287    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:36.638697 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:36.638661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:40.319548 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.316457    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"]
Sep 09 12:56:40.319548 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.317232    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be379105-d981-4d02-ba1e-de25ebe72463" containerName="discovery"
Sep 09 12:56:40.319548 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.317257    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be379105-d981-4d02-ba1e-de25ebe72463" containerName="discovery"
Sep 09 12:56:40.319548 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.317433    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be379105-d981-4d02-ba1e-de25ebe72463" containerName="discovery"
Sep 09 12:56:40.323587 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.323558    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.325140 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.325114    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"]
Sep 09 12:56:40.325802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.325782    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 09 12:56:40.325917 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.325782    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 09 12:56:40.325917 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.325862    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wclvq\""
Sep 09 12:56:40.367798 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.367773    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.367949 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.367804    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.367949 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.367826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78ml4\" (UniqueName: \"kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.412172 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.412148    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"]
Sep 09 12:56:40.414569 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.414552    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.422867 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.422844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"]
Sep 09 12:56:40.468955 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.468929    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.469067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.468961    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.469067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.468980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78ml4\" (UniqueName: \"kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.469067 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.469010    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.469218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.469080    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw58h\" (UniqueName: \"kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.469218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.469105    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.469310 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.469288    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.469346 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.469312    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.476990 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.476970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78ml4\" (UniqueName: \"kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.512708 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.512682    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"]
Sep 09 12:56:40.515219 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.515205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.525378 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.525358    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"]
Sep 09 12:56:40.570510 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570441    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw58h\" (UniqueName: \"kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.570510 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.570653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zx4rm\" (UniqueName: \"kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.570653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570566    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.570653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570609    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.570653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.570841 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.570907 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.570892    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.578356 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.578329    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw58h\" (UniqueName: \"kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.612739 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.612713    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"]
Sep 09 12:56:40.615274 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.615259    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.624290 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.624263    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"]
Sep 09 12:56:40.633183 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.633163    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:40.671205 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671175    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.671350 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.671350 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671268    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zx4rm\" (UniqueName: \"kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.671350 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671290    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.671493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.671493 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671392    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdzlf\" (UniqueName: \"kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.671677 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.671744 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.671713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.681597 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.681391    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zx4rm\" (UniqueName: \"kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.723860 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.723828    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:40.772186 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.772155    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.772286 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.772256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdzlf\" (UniqueName: \"kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.772350 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.772320    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.773458 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.772823    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.773458 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.773091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.781941 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.781895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdzlf\" (UniqueName: \"kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.824643 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.824583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:40.901707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.901682    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"]
Sep 09 12:56:40.903900 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:40.903871    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2aaf0e78_d210_4904_aa3c_05365a7e4e76.slice/crio-eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954 WatchSource:0}: Error finding container eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954: Status 404 returned error can't find the container with id eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954
Sep 09 12:56:40.924948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.924923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:40.998906 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:40.998884    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"]
Sep 09 12:56:41.000687 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:41.000661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabe74905_2c75_46b3_a1d3_cfce5f981c18.slice/crio-199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf WatchSource:0}: Error finding container 199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf: Status 404 returned error can't find the container with id 199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf
Sep 09 12:56:41.092729 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.092707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"]
Sep 09 12:56:41.094543 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:41.094513    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5eb254_87fa_4a36_be62_114ed99a80f9.slice/crio-6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852 WatchSource:0}: Error finding container 6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852: Status 404 returned error can't find the container with id 6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852
Sep 09 12:56:41.199125 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.198990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"]
Sep 09 12:56:41.201870 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:41.201837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca1740ed_a701_46d7_85f5_1310bf625dc1.slice/crio-f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113 WatchSource:0}: Error finding container f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113: Status 404 returned error can't find the container with id f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113
Sep 09 12:56:41.691676 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.691637    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerID="555abd97d1201843eb778a60ceec381c255df9dbc16692c4bffbd6539a22282d" exitCode=0
Sep 09 12:56:41.692149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.691722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8" event={"ID":"1e5eb254-87fa-4a36-be62-114ed99a80f9","Type":"ContainerDied","Data":"555abd97d1201843eb778a60ceec381c255df9dbc16692c4bffbd6539a22282d"}
Sep 09 12:56:41.692149 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.691768    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8" event={"ID":"1e5eb254-87fa-4a36-be62-114ed99a80f9","Type":"ContainerStarted","Data":"6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852"}
Sep 09 12:56:41.693301 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.693233    2580 generic.go:356] "Generic (PLEG): container finished" podID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerID="2d14981e71c268b250370e2fbfc401c4ae36454732080e4b79547158269f87df" exitCode=0
Sep 09 12:56:41.693301 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.693282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh" event={"ID":"abe74905-2c75-46b3-a1d3-cfce5f981c18","Type":"ContainerDied","Data":"2d14981e71c268b250370e2fbfc401c4ae36454732080e4b79547158269f87df"}
Sep 09 12:56:41.693623 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.693330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh" event={"ID":"abe74905-2c75-46b3-a1d3-cfce5f981c18","Type":"ContainerStarted","Data":"199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf"}
Sep 09 12:56:41.695082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.695058    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerID="27b7c0b93789409f5c48b5e6c6e187a95ffae17ec2ae0bb7e551790273ee1663" exitCode=0
Sep 09 12:56:41.695176 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.695127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4" event={"ID":"ca1740ed-a701-46d7-85f5-1310bf625dc1","Type":"ContainerDied","Data":"27b7c0b93789409f5c48b5e6c6e187a95ffae17ec2ae0bb7e551790273ee1663"}
Sep 09 12:56:41.695176 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.695161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4" event={"ID":"ca1740ed-a701-46d7-85f5-1310bf625dc1","Type":"ContainerStarted","Data":"f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113"}
Sep 09 12:56:41.696921 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.696903    2580 generic.go:356] "Generic (PLEG): container finished" podID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerID="2e2d3f6f9f2a0b80ad8fe210f2b8e2ec05dbf69145f7abb3fc80fb8f3c930c7a" exitCode=0
Sep 09 12:56:41.697007 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.696959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk" event={"ID":"2aaf0e78-d210-4904-aa3c-05365a7e4e76","Type":"ContainerDied","Data":"2e2d3f6f9f2a0b80ad8fe210f2b8e2ec05dbf69145f7abb3fc80fb8f3c930c7a"}
Sep 09 12:56:41.697007 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:41.696987    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk" event={"ID":"2aaf0e78-d210-4904-aa3c-05365a7e4e76","Type":"ContainerStarted","Data":"eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954"}
Sep 09 12:56:42.704207 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.704171    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerID="94b6d485ec1f6a6c8b580742211cf17badb3d35a1ba2c35e3be9a814eb657419" exitCode=0
Sep 09 12:56:42.704642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.704253    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8" event={"ID":"1e5eb254-87fa-4a36-be62-114ed99a80f9","Type":"ContainerDied","Data":"94b6d485ec1f6a6c8b580742211cf17badb3d35a1ba2c35e3be9a814eb657419"}
Sep 09 12:56:42.706106 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.706085    2580 generic.go:356] "Generic (PLEG): container finished" podID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerID="d4946e9aad340559b71fcc994aa1e8105c2a0fdb41749c53d938e156c20f5c71" exitCode=0
Sep 09 12:56:42.706212 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.706163    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh" event={"ID":"abe74905-2c75-46b3-a1d3-cfce5f981c18","Type":"ContainerDied","Data":"d4946e9aad340559b71fcc994aa1e8105c2a0fdb41749c53d938e156c20f5c71"}
Sep 09 12:56:42.708030 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.708011    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerID="76c3fe03ac83017d806440ed85627bc0ea5ff811e57009c81b9ef0314ac6881a" exitCode=0
Sep 09 12:56:42.708126 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:42.708044    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4" event={"ID":"ca1740ed-a701-46d7-85f5-1310bf625dc1","Type":"ContainerDied","Data":"76c3fe03ac83017d806440ed85627bc0ea5ff811e57009c81b9ef0314ac6881a"}
Sep 09 12:56:43.715094 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.715063    2580 generic.go:356] "Generic (PLEG): container finished" podID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerID="62c7f584f44d93d15c64e91aa5c02fddcfe885d61eaac68f73faca482cf4c035" exitCode=0
Sep 09 12:56:43.715474 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.715160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh" event={"ID":"abe74905-2c75-46b3-a1d3-cfce5f981c18","Type":"ContainerDied","Data":"62c7f584f44d93d15c64e91aa5c02fddcfe885d61eaac68f73faca482cf4c035"}
Sep 09 12:56:43.717103 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.717080    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerID="490fe01de98fc13aace9a791e7504f4ed9d2322972ad40eb4733da90b03049df" exitCode=0
Sep 09 12:56:43.717208 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.717125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4" event={"ID":"ca1740ed-a701-46d7-85f5-1310bf625dc1","Type":"ContainerDied","Data":"490fe01de98fc13aace9a791e7504f4ed9d2322972ad40eb4733da90b03049df"}
Sep 09 12:56:43.718791 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.718772    2580 generic.go:356] "Generic (PLEG): container finished" podID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerID="e32b896ccbaa6248c69455f4e05a1634ecc86a7c3e95f478e35820e871f3603d" exitCode=0
Sep 09 12:56:43.718870 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.718849    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk" event={"ID":"2aaf0e78-d210-4904-aa3c-05365a7e4e76","Type":"ContainerDied","Data":"e32b896ccbaa6248c69455f4e05a1634ecc86a7c3e95f478e35820e871f3603d"}
Sep 09 12:56:43.721139 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.721122    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerID="ba7bc045f4149ddc7fe236d3e102ed089e552a2b7efcd546ff5b64b240489fa1" exitCode=0
Sep 09 12:56:43.721214 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:43.721146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8" event={"ID":"1e5eb254-87fa-4a36-be62-114ed99a80f9","Type":"ContainerDied","Data":"ba7bc045f4149ddc7fe236d3e102ed089e552a2b7efcd546ff5b64b240489fa1"}
Sep 09 12:56:44.728203 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.728168    2580 generic.go:356] "Generic (PLEG): container finished" podID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerID="7bc0ce17c59cc4995478aab90ceb48256a02cf5c8871b876b208aacb2f2d5363" exitCode=0
Sep 09 12:56:44.728627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.728224    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk" event={"ID":"2aaf0e78-d210-4904-aa3c-05365a7e4e76","Type":"ContainerDied","Data":"7bc0ce17c59cc4995478aab90ceb48256a02cf5c8871b876b208aacb2f2d5363"}
Sep 09 12:56:44.867318 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.867294    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:44.909301 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.909272    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util\") pod \"1e5eb254-87fa-4a36-be62-114ed99a80f9\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") "
Sep 09 12:56:44.909428 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.909355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle\") pod \"1e5eb254-87fa-4a36-be62-114ed99a80f9\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") "
Sep 09 12:56:44.909428 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.909418    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zx4rm\" (UniqueName: \"kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm\") pod \"1e5eb254-87fa-4a36-be62-114ed99a80f9\" (UID: \"1e5eb254-87fa-4a36-be62-114ed99a80f9\") "
Sep 09 12:56:44.909859 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.909834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle" (OuterVolumeSpecName: "bundle") pod "1e5eb254-87fa-4a36-be62-114ed99a80f9" (UID: "1e5eb254-87fa-4a36-be62-114ed99a80f9"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:44.911774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.911754    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm" (OuterVolumeSpecName: "kube-api-access-zx4rm") pod "1e5eb254-87fa-4a36-be62-114ed99a80f9" (UID: "1e5eb254-87fa-4a36-be62-114ed99a80f9"). InnerVolumeSpecName "kube-api-access-zx4rm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:56:44.915696 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.915671    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util" (OuterVolumeSpecName: "util") pod "1e5eb254-87fa-4a36-be62-114ed99a80f9" (UID: "1e5eb254-87fa-4a36-be62-114ed99a80f9"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:44.917662 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.917640    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:44.920853 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:44.920832    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:45.010601 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010524    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle\") pod \"abe74905-2c75-46b3-a1d3-cfce5f981c18\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") "
Sep 09 12:56:45.010601 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010561    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util\") pod \"abe74905-2c75-46b3-a1d3-cfce5f981c18\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") "
Sep 09 12:56:45.010601 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010583    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdzlf\" (UniqueName: \"kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf\") pod \"ca1740ed-a701-46d7-85f5-1310bf625dc1\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") "
Sep 09 12:56:45.010862 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010663    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle\") pod \"ca1740ed-a701-46d7-85f5-1310bf625dc1\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") "
Sep 09 12:56:45.010862 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010691    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util\") pod \"ca1740ed-a701-46d7-85f5-1310bf625dc1\" (UID: \"ca1740ed-a701-46d7-85f5-1310bf625dc1\") "
Sep 09 12:56:45.010862 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pw58h\" (UniqueName: \"kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h\") pod \"abe74905-2c75-46b3-a1d3-cfce5f981c18\" (UID: \"abe74905-2c75-46b3-a1d3-cfce5f981c18\") "
Sep 09 12:56:45.011079 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010976    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zx4rm\" (UniqueName: \"kubernetes.io/projected/1e5eb254-87fa-4a36-be62-114ed99a80f9-kube-api-access-zx4rm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.011079 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.010995    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.011079 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.011008    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1e5eb254-87fa-4a36-be62-114ed99a80f9-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.011223 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.011166    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle" (OuterVolumeSpecName: "bundle") pod "abe74905-2c75-46b3-a1d3-cfce5f981c18" (UID: "abe74905-2c75-46b3-a1d3-cfce5f981c18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:45.011449 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.011428    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle" (OuterVolumeSpecName: "bundle") pod "ca1740ed-a701-46d7-85f5-1310bf625dc1" (UID: "ca1740ed-a701-46d7-85f5-1310bf625dc1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:45.012802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.012777    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf" (OuterVolumeSpecName: "kube-api-access-qdzlf") pod "ca1740ed-a701-46d7-85f5-1310bf625dc1" (UID: "ca1740ed-a701-46d7-85f5-1310bf625dc1"). InnerVolumeSpecName "kube-api-access-qdzlf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:56:45.013199 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.013175    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h" (OuterVolumeSpecName: "kube-api-access-pw58h") pod "abe74905-2c75-46b3-a1d3-cfce5f981c18" (UID: "abe74905-2c75-46b3-a1d3-cfce5f981c18"). InnerVolumeSpecName "kube-api-access-pw58h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:56:45.016204 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.016181    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util" (OuterVolumeSpecName: "util") pod "ca1740ed-a701-46d7-85f5-1310bf625dc1" (UID: "ca1740ed-a701-46d7-85f5-1310bf625dc1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:45.016684 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.016666    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util" (OuterVolumeSpecName: "util") pod "abe74905-2c75-46b3-a1d3-cfce5f981c18" (UID: "abe74905-2c75-46b3-a1d3-cfce5f981c18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:45.112218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112193    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.112218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112214    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ca1740ed-a701-46d7-85f5-1310bf625dc1-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.112349 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112224    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pw58h\" (UniqueName: \"kubernetes.io/projected/abe74905-2c75-46b3-a1d3-cfce5f981c18-kube-api-access-pw58h\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.112349 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112235    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.112349 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112243    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/abe74905-2c75-46b3-a1d3-cfce5f981c18-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.112349 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.112250    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdzlf\" (UniqueName: \"kubernetes.io/projected/ca1740ed-a701-46d7-85f5-1310bf625dc1-kube-api-access-qdzlf\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:45.734909 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.734879    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8"
Sep 09 12:56:45.735393 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.734875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c62xls8" event={"ID":"1e5eb254-87fa-4a36-be62-114ed99a80f9","Type":"ContainerDied","Data":"6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852"}
Sep 09 12:56:45.735393 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.735002    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6fb2c4d95f82d9937192b6891b5fafcbc3295c2ed8f7883272d8be7970ac1852"
Sep 09 12:56:45.736780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.736754    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh"
Sep 09 12:56:45.736912 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.736805    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c46hzh" event={"ID":"abe74905-2c75-46b3-a1d3-cfce5f981c18","Type":"ContainerDied","Data":"199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf"}
Sep 09 12:56:45.736912 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.736847    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="199fc24f1c9d942460964639ac868522318733e1c2a5292397d5ec44ed6d1faf"
Sep 09 12:56:45.738838 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.738821    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4"
Sep 09 12:56:45.738978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.738858    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847at2lw4" event={"ID":"ca1740ed-a701-46d7-85f5-1310bf625dc1","Type":"ContainerDied","Data":"f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113"}
Sep 09 12:56:45.738978 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.738891    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f824c25cec3213861f2dd232638fb918872321b385d3a08b87b3299da1be8113"
Sep 09 12:56:45.866052 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.866030    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:45.919036 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.919010    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util\") pod \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") "
Sep 09 12:56:45.919183 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.919053    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle\") pod \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") "
Sep 09 12:56:45.919183 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.919108    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-78ml4\" (UniqueName: \"kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4\") pod \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\" (UID: \"2aaf0e78-d210-4904-aa3c-05365a7e4e76\") "
Sep 09 12:56:45.919569 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.919544    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle" (OuterVolumeSpecName: "bundle") pod "2aaf0e78-d210-4904-aa3c-05365a7e4e76" (UID: "2aaf0e78-d210-4904-aa3c-05365a7e4e76"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:45.921121 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.921092    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4" (OuterVolumeSpecName: "kube-api-access-78ml4") pod "2aaf0e78-d210-4904-aa3c-05365a7e4e76" (UID: "2aaf0e78-d210-4904-aa3c-05365a7e4e76"). InnerVolumeSpecName "kube-api-access-78ml4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:56:45.924394 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:45.924361    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util" (OuterVolumeSpecName: "util") pod "2aaf0e78-d210-4904-aa3c-05365a7e4e76" (UID: "2aaf0e78-d210-4904-aa3c-05365a7e4e76"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:56:46.020192 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.020124    2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-util\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:46.020192 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.020149    2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2aaf0e78-d210-4904-aa3c-05365a7e4e76-bundle\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:46.020192 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.020161    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-78ml4\" (UniqueName: \"kubernetes.io/projected/2aaf0e78-d210-4904-aa3c-05365a7e4e76-kube-api-access-78ml4\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:56:46.746177 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.746090    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk" event={"ID":"2aaf0e78-d210-4904-aa3c-05365a7e4e76","Type":"ContainerDied","Data":"eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954"}
Sep 09 12:56:46.746177 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.746131    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7srkrk"
Sep 09 12:56:46.746666 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:46.746138    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="eb638513ee235ebee9cb3a173ce28eb211fe1bee38edfa1fc02a51f3a147d954"
Sep 09 12:56:50.870290 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:50.870256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:52.919246 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919212    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"]
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919578    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="extract"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919591    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="extract"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919601    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="pull"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919608    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="pull"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919619    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="util"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919624    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="util"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919636    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="util"
Sep 09 12:56:52.919637 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919641    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="util"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919647    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919653    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919661    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919666    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919675    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919680    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="pull"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919689    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="util"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919694    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="util"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919699    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919704    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919709    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919713    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919720    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919725    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919729    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="util"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919735    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="util"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919783    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2aaf0e78-d210-4904-aa3c-05365a7e4e76" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919792    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca1740ed-a701-46d7-85f5-1310bf625dc1" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919799    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5eb254-87fa-4a36-be62-114ed99a80f9" containerName="extract"
Sep 09 12:56:52.919895 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.919805    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abe74905-2c75-46b3-a1d3-cfce5f981c18" containerName="extract"
Sep 09 12:56:52.922723 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.922706    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:52.924990 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.924969    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 12:56:52.925116 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.925044    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 12:56:52.925850 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.925829    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-d979c\""
Sep 09 12:56:52.935955 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:52.935937    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"]
Sep 09 12:56:53.075434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.075407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzq7x\" (UniqueName: \"kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x\") pod \"limitador-operator-controller-manager-74495c69f8-c9782\" (UID: \"ee012789-6695-4d29-9764-effd4ef46c84\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:53.176332 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.176255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzq7x\" (UniqueName: \"kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x\") pod \"limitador-operator-controller-manager-74495c69f8-c9782\" (UID: \"ee012789-6695-4d29-9764-effd4ef46c84\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:53.186913 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.186883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzq7x\" (UniqueName: \"kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x\") pod \"limitador-operator-controller-manager-74495c69f8-c9782\" (UID: \"ee012789-6695-4d29-9764-effd4ef46c84\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:53.232927 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.232902    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:53.511561 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.511532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"]
Sep 09 12:56:53.513561 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:53.513531    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee012789_6695_4d29_9764_effd4ef46c84.slice/crio-7992be2e6c7c7bcff80862d4645d9a98c0ced51d16d6cf40e43d4ecec806bf35 WatchSource:0}: Error finding container 7992be2e6c7c7bcff80862d4645d9a98c0ced51d16d6cf40e43d4ecec806bf35: Status 404 returned error can't find the container with id 7992be2e6c7c7bcff80862d4645d9a98c0ced51d16d6cf40e43d4ecec806bf35
Sep 09 12:56:53.714007 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.713967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:53.779659 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:53.779572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782" event={"ID":"ee012789-6695-4d29-9764-effd4ef46c84","Type":"ContainerStarted","Data":"7992be2e6c7c7bcff80862d4645d9a98c0ced51d16d6cf40e43d4ecec806bf35"}
Sep 09 12:56:55.018174 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.018147    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"]
Sep 09 12:56:55.022794 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.022774    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:55.024974 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.024950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 12:56:55.025071 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.025013    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-6gqf9\""
Sep 09 12:56:55.031315 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.031291    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"]
Sep 09 12:56:55.093057 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.093032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jplbd\" (UniqueName: \"kubernetes.io/projected/f5f610ea-dba0-4363-8f43-1f345023176a-kube-api-access-jplbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-5tkgp\" (UID: \"f5f610ea-dba0-4363-8f43-1f345023176a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:55.194412 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.194378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jplbd\" (UniqueName: \"kubernetes.io/projected/f5f610ea-dba0-4363-8f43-1f345023176a-kube-api-access-jplbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-5tkgp\" (UID: \"f5f610ea-dba0-4363-8f43-1f345023176a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:55.202300 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.202273    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jplbd\" (UniqueName: \"kubernetes.io/projected/f5f610ea-dba0-4363-8f43-1f345023176a-kube-api-access-jplbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-5tkgp\" (UID: \"f5f610ea-dba0-4363-8f43-1f345023176a\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:55.268543 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.268449    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:55.333653 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.333631    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:55.604998 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.604921    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"]
Sep 09 12:56:55.607845 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:55.607658    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5f610ea_dba0_4363_8f43_1f345023176a.slice/crio-4d25a89fc44461de35bf126b5fae19d7d58274b7a893f1e8771ac2c1dbb14cf9 WatchSource:0}: Error finding container 4d25a89fc44461de35bf126b5fae19d7d58274b7a893f1e8771ac2c1dbb14cf9: Status 404 returned error can't find the container with id 4d25a89fc44461de35bf126b5fae19d7d58274b7a893f1e8771ac2c1dbb14cf9
Sep 09 12:56:55.669627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.669594    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:55.794963 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:55.794920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp" event={"ID":"f5f610ea-dba0-4363-8f43-1f345023176a","Type":"ContainerStarted","Data":"4d25a89fc44461de35bf126b5fae19d7d58274b7a893f1e8771ac2c1dbb14cf9"}
Sep 09 12:56:56.009216 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:56.009185    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:57.310227 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.310180    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:57.727578 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.727541    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-pmb8j"]
Sep 09 12:56:57.731176 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.731155    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:56:57.734083 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.734057    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-p69n7\""
Sep 09 12:56:57.744444 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.744419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-pmb8j"]
Sep 09 12:56:57.807173 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.807134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782" event={"ID":"ee012789-6695-4d29-9764-effd4ef46c84","Type":"ContainerStarted","Data":"59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a"}
Sep 09 12:56:57.807355 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.807287    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:56:57.823058 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.823022    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d77z8\" (UniqueName: \"kubernetes.io/projected/9e0d4df1-2852-4075-a8a8-0280475be7f3-kube-api-access-d77z8\") pod \"authorino-operator-76b478bd78-pmb8j\" (UID: \"9e0d4df1-2852-4075-a8a8-0280475be7f3\") " pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:56:57.841353 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.841311    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782" podStartSLOduration=2.180516185 podStartE2EDuration="5.841296772s" totalImagesPullingTime="3.660780587s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:56:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:56:57.838758248 +0000 UTC m=+515.874006155" watchObservedRunningTime="2026-09-09 12:56:57.841296772 +0000 UTC m=+515.876544680"
Sep 09 12:56:57.924663 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.924627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d77z8\" (UniqueName: \"kubernetes.io/projected/9e0d4df1-2852-4075-a8a8-0280475be7f3-kube-api-access-d77z8\") pod \"authorino-operator-76b478bd78-pmb8j\" (UID: \"9e0d4df1-2852-4075-a8a8-0280475be7f3\") " pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:56:57.935970 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:57.935942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d77z8\" (UniqueName: \"kubernetes.io/projected/9e0d4df1-2852-4075-a8a8-0280475be7f3-kube-api-access-d77z8\") pod \"authorino-operator-76b478bd78-pmb8j\" (UID: \"9e0d4df1-2852-4075-a8a8-0280475be7f3\") " pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:56:58.044700 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.044615    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:56:58.290951 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.290910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:58.394757 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.394721    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:58.519225 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.519202    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-pmb8j"]
Sep 09 12:56:58.520845 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:56:58.520817    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e0d4df1_2852_4075_a8a8_0280475be7f3.slice/crio-9f71de159d34ce7c3d6571216e990f8bfa36165595152608f630f3f30207ae24 WatchSource:0}: Error finding container 9f71de159d34ce7c3d6571216e990f8bfa36165595152608f630f3f30207ae24: Status 404 returned error can't find the container with id 9f71de159d34ce7c3d6571216e990f8bfa36165595152608f630f3f30207ae24
Sep 09 12:56:58.700989 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.700955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:58.794982 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.794950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:58.814380 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.814350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp" event={"ID":"f5f610ea-dba0-4363-8f43-1f345023176a","Type":"ContainerStarted","Data":"03d3ff72972d70dab9c18a77a374c84ee23a511aa5927a78e69fb17f5736cc99"}
Sep 09 12:56:58.814489 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.814413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:56:58.815733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.815714    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j" event={"ID":"9e0d4df1-2852-4075-a8a8-0280475be7f3","Type":"ContainerStarted","Data":"9f71de159d34ce7c3d6571216e990f8bfa36165595152608f630f3f30207ae24"}
Sep 09 12:56:58.834446 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:58.834404    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp" podStartSLOduration=1.140532182 podStartE2EDuration="3.834392531s" totalImagesPullingTime="2.693860349s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:56:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:56:58.832327124 +0000 UTC m=+516.867575031" watchObservedRunningTime="2026-09-09 12:56:58.834392531 +0000 UTC m=+516.869640438"
Sep 09 12:56:59.103600 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:59.103522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:59.213114 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:59.213084    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:59.723412 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:59.723358    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:56:59.817593 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:56:59.817559    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:00.829109 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:00.829078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j" event={"ID":"9e0d4df1-2852-4075-a8a8-0280475be7f3","Type":"ContainerStarted","Data":"52fb49676015240122667cc7cc7a9e008913da739e8f7ac898263ac7d37b6144"}
Sep 09 12:57:00.829546 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:00.829213    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:57:00.849804 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:00.849762    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j" podStartSLOduration=2.454661119 podStartE2EDuration="3.849748019s" totalImagesPullingTime="1.3950869s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:00.847690724 +0000 UTC m=+518.882938631" watchObservedRunningTime="2026-09-09 12:57:00.849748019 +0000 UTC m=+518.884995926"
Sep 09 12:57:02.079871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.079838    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"]
Sep 09 12:57:02.083303 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.083286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.087110 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.087087    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lfmkw\""
Sep 09 12:57:02.094814 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.094791    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"]
Sep 09 12:57:02.160700 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.160673    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.160852 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.160723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g79dw\" (UniqueName: \"kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.261225 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.261197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.261354 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.261246    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g79dw\" (UniqueName: \"kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.261652 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.261635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.268926 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.268901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g79dw\" (UniqueName: \"kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.392863 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.392843    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:02.674201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.674171    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"]
Sep 09 12:57:02.675913 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:02.675867    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b18c1e_e425_40c4_b290_ad275a742ce6.slice/crio-ea8b1f058b2308276c7d60108b569fc1958bbc3d35ecb4201188152556b2e100 WatchSource:0}: Error finding container ea8b1f058b2308276c7d60108b569fc1958bbc3d35ecb4201188152556b2e100: Status 404 returned error can't find the container with id ea8b1f058b2308276c7d60108b569fc1958bbc3d35ecb4201188152556b2e100
Sep 09 12:57:02.840525 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:02.840470    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb" event={"ID":"32b18c1e-e425-40c4-b290-ad275a742ce6","Type":"ContainerStarted","Data":"ea8b1f058b2308276c7d60108b569fc1958bbc3d35ecb4201188152556b2e100"}
Sep 09 12:57:03.479275 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:03.479232    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:03.583939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:03.583901    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:06.502838 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:06.502798    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:06.559658 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:06.559619    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:06.867994 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:06.867914    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb" event={"ID":"32b18c1e-e425-40c4-b290-ad275a742ce6","Type":"ContainerStarted","Data":"a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b"}
Sep 09 12:57:06.867994 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:06.867988    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:06.889492 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:06.889436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb" podStartSLOduration=0.988926733 podStartE2EDuration="4.889420836s" totalImagesPullingTime="3.900494103s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:06.887122138 +0000 UTC m=+524.922370045" watchObservedRunningTime="2026-09-09 12:57:06.889420836 +0000 UTC m=+524.924668742"
Sep 09 12:57:08.818255 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:08.818223    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:09.509027 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:09.508986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:09.659021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:09.658973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:09.824909 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:09.824835    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5tkgp"
Sep 09 12:57:11.835959 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:11.835932    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-pmb8j"
Sep 09 12:57:12.259080 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:12.259040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:12.400855 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:12.400819    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:14.729298 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:14.729266    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:16.820688 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:16.820649    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:16.920200 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:16.920156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:17.875334 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:17.875305    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:18.949603 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:18.949571    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"]
Sep 09 12:57:18.952803 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:18.952786    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:18.967094 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:18.967072    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"]
Sep 09 12:57:18.990055 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:18.990030    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npz6\" (UniqueName: \"kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:18.990194 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:18.990079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.090733 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.090708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npz6\" (UniqueName: \"kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.090848 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.090757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.091084 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.091067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.102130 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.102101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npz6\" (UniqueName: \"kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.263096 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.263006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.541656 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.541628    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"]
Sep 09 12:57:19.542954 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:19.542929    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod568a2d45_ffcc_4f88_b880_00db15b95298.slice/crio-ce4ec38ee03d983804c56c459ec231b19a23ca122d75de733d0663f7fa829de4 WatchSource:0}: Error finding container ce4ec38ee03d983804c56c459ec231b19a23ca122d75de733d0663f7fa829de4: Status 404 returned error can't find the container with id ce4ec38ee03d983804c56c459ec231b19a23ca122d75de733d0663f7fa829de4
Sep 09 12:57:19.938569 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.938533    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz" event={"ID":"568a2d45-ffcc-4f88-b880-00db15b95298","Type":"ContainerStarted","Data":"c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b"}
Sep 09 12:57:19.938736 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.938589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz" event={"ID":"568a2d45-ffcc-4f88-b880-00db15b95298","Type":"ContainerStarted","Data":"ce4ec38ee03d983804c56c459ec231b19a23ca122d75de733d0663f7fa829de4"}
Sep 09 12:57:19.938736 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.938649    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:19.960552 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:19.960492    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz" podStartSLOduration=1.9604778870000001 podStartE2EDuration="1.960477887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:19.957081495 +0000 UTC m=+537.992329403" watchObservedRunningTime="2026-09-09 12:57:19.960477887 +0000 UTC m=+537.995725829"
Sep 09 12:57:20.374287 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.374206    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"]
Sep 09 12:57:20.383869 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.383843    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"]
Sep 09 12:57:20.394336 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.394308    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:20.394691 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.394678    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" containerName="manager"
Sep 09 12:57:20.394742 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.394694    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" containerName="manager"
Sep 09 12:57:20.394801 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.394790    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" containerName="manager"
Sep 09 12:57:20.398342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.398322    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.399289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.399265    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"]
Sep 09 12:57:20.399455 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.399431    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" containerName="manager" containerID="cri-o://a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b" gracePeriod=2
Sep 09 12:57:20.399573 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.399554    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nskk4\" (UniqueName: \"kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.399654 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.399633    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.414924 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.414858    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"]
Sep 09 12:57:20.420774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.420748    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:20.440821 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.440782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"]
Sep 09 12:57:20.441144 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.441096    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782" podUID="ee012789-6695-4d29-9764-effd4ef46c84" containerName="manager" containerID="cri-o://59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a" gracePeriod=2
Sep 09 12:57:20.454185 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.453360    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.456789 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.456751    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-j6pmb\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:20.458373 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.458351    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:20.458802 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.458784    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" containerName="manager"
Sep 09 12:57:20.458940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.458920    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" containerName="manager"
Sep 09 12:57:20.459082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.459066    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" containerName="manager"
Sep 09 12:57:20.462667 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.462610    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"]
Sep 09 12:57:20.462780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.462697    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:20.462780 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.462711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.463194 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.463175    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"]
Sep 09 12:57:20.463475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.463465    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee012789-6695-4d29-9764-effd4ef46c84" containerName="manager"
Sep 09 12:57:20.463537 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.463477    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee012789-6695-4d29-9764-effd4ef46c84" containerName="manager"
Sep 09 12:57:20.463579 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.463567    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee012789-6695-4d29-9764-effd4ef46c84" containerName="manager"
Sep 09 12:57:20.467568 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.467550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:20.477826 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.477806    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"]
Sep 09 12:57:20.487918 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.487892    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.490279 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.490246    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.500432 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.500410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nskk4\" (UniqueName: \"kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.500556 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.500473    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.500556 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.500521    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd5f5\" (UniqueName: \"kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.500655 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.500597    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.500942 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.500923    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.518002 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.517972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nskk4\" (UniqueName: \"kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4\") pod \"kuadrant-operator-controller-manager-67c5dcb48c-s442g\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.601379 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.601350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cd5f5\" (UniqueName: \"kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.601379 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.601392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.601680 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.601463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p98b7\" (UniqueName: \"kubernetes.io/projected/d60e23d2-c572-4c93-ab81-59f477b7e4b2-kube-api-access-p98b7\") pod \"limitador-operator-controller-manager-74495c69f8-4v6zj\" (UID: \"d60e23d2-c572-4c93-ab81-59f477b7e4b2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:20.601874 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.601854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.613102 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.613074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd5f5\" (UniqueName: \"kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5\") pod \"kuadrant-operator-controller-manager-76f8f686bf-fpzvl\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.670576 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.670554    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:20.672684 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.672654    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.680084 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.680066    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.682013 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.681988    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.702405 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.702386    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jzq7x\" (UniqueName: \"kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x\") pod \"ee012789-6695-4d29-9764-effd4ef46c84\" (UID: \"ee012789-6695-4d29-9764-effd4ef46c84\") "
Sep 09 12:57:20.702528 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.702416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g79dw\" (UniqueName: \"kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw\") pod \"32b18c1e-e425-40c4-b290-ad275a742ce6\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") "
Sep 09 12:57:20.702528 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.702449    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume\") pod \"32b18c1e-e425-40c4-b290-ad275a742ce6\" (UID: \"32b18c1e-e425-40c4-b290-ad275a742ce6\") "
Sep 09 12:57:20.702734 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.702679    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p98b7\" (UniqueName: \"kubernetes.io/projected/d60e23d2-c572-4c93-ab81-59f477b7e4b2-kube-api-access-p98b7\") pod \"limitador-operator-controller-manager-74495c69f8-4v6zj\" (UID: \"d60e23d2-c572-4c93-ab81-59f477b7e4b2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:20.703013 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.702991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "32b18c1e-e425-40c4-b290-ad275a742ce6" (UID: "32b18c1e-e425-40c4-b290-ad275a742ce6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:57:20.704660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.704641    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x" (OuterVolumeSpecName: "kube-api-access-jzq7x") pod "ee012789-6695-4d29-9764-effd4ef46c84" (UID: "ee012789-6695-4d29-9764-effd4ef46c84"). InnerVolumeSpecName "kube-api-access-jzq7x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:57:20.704725 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.704667    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw" (OuterVolumeSpecName: "kube-api-access-g79dw") pod "32b18c1e-e425-40c4-b290-ad275a742ce6" (UID: "32b18c1e-e425-40c4-b290-ad275a742ce6"). InnerVolumeSpecName "kube-api-access-g79dw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:57:20.709942 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.709924    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p98b7\" (UniqueName: \"kubernetes.io/projected/d60e23d2-c572-4c93-ab81-59f477b7e4b2-kube-api-access-p98b7\") pod \"limitador-operator-controller-manager-74495c69f8-4v6zj\" (UID: \"d60e23d2-c572-4c93-ab81-59f477b7e4b2\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:20.710062 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.710026    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:20.803164 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.803137    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jzq7x\" (UniqueName: \"kubernetes.io/projected/ee012789-6695-4d29-9764-effd4ef46c84-kube-api-access-jzq7x\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:20.803164 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.803162    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g79dw\" (UniqueName: \"kubernetes.io/projected/32b18c1e-e425-40c4-b290-ad275a742ce6-kube-api-access-g79dw\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:20.803341 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.803174    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/32b18c1e-e425-40c4-b290-ad275a742ce6-extensions-socket-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:20.839104 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.839081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:20.844233 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.844205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:20.946273 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.946223    2580 generic.go:356] "Generic (PLEG): container finished" podID="ee012789-6695-4d29-9764-effd4ef46c84" containerID="59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a" exitCode=0
Sep 09 12:57:20.946273 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.946267    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.946475 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.946294    2580 scope.go:117] "RemoveContainer" containerID="59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a"
Sep 09 12:57:20.948132 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.948106    2580 generic.go:356] "Generic (PLEG): container finished" podID="32b18c1e-e425-40c4-b290-ad275a742ce6" containerID="a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b" exitCode=0
Sep 09 12:57:20.948254 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.948147    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-j6pmb"
Sep 09 12:57:20.948613 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.948591    2580 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz" container="manager"
Sep 09 12:57:20.948785 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.948756    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.951054 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.951016    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.957633 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.957616    2580 scope.go:117] "RemoveContainer" containerID="59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a"
Sep 09 12:57:20.957891 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.957871    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a\": container with ID starting with 59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a not found: ID does not exist" containerID="59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a"
Sep 09 12:57:20.957934 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.957898    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a"} err="failed to get container status \"59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a\": rpc error: code = NotFound desc = could not find container \"59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a\": container with ID starting with 59bef2b4a5a81b5ea6bdd12911bdafd7814b11a667c4f385f8a2137085251a1a not found: ID does not exist"
Sep 09 12:57:20.957934 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.957914    2580 scope.go:117] "RemoveContainer" containerID="a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b"
Sep 09 12:57:20.961049 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.961026    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.963090 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.963066    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:20.966644 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.966626    2580 scope.go:117] "RemoveContainer" containerID="a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b"
Sep 09 12:57:20.966911 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:20.966891    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b\": container with ID starting with a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b not found: ID does not exist" containerID="a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b"
Sep 09 12:57:20.966971 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.966917    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b"} err="failed to get container status \"a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b\": rpc error: code = NotFound desc = could not find container \"a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b\": container with ID starting with a6ba84efda2a43d27030bff1eb00e95e5d5efb8abe9c828dc3b8d7daec5c8f0b not found: ID does not exist"
Sep 09 12:57:20.996743 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:20.996726    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:20.998335 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:20.998310    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode13a1c01_feb5_4d3b_9b5f_36585128465f.slice/crio-25fffe76973f9284cfbb27c0ecd09015edb11cf899fb508d4ac150b7c1557211 WatchSource:0}: Error finding container 25fffe76973f9284cfbb27c0ecd09015edb11cf899fb508d4ac150b7c1557211: Status 404 returned error can't find the container with id 25fffe76973f9284cfbb27c0ecd09015edb11cf899fb508d4ac150b7c1557211
Sep 09 12:57:21.119185 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.119150    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"]
Sep 09 12:57:21.120651 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:21.120623    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd60e23d2_c572_4c93_ab81_59f477b7e4b2.slice/crio-3108208ddea3b95aa7565ba82b481feb90fb040ba9dae1d686417999f6d4d579 WatchSource:0}: Error finding container 3108208ddea3b95aa7565ba82b481feb90fb040ba9dae1d686417999f6d4d579: Status 404 returned error can't find the container with id 3108208ddea3b95aa7565ba82b481feb90fb040ba9dae1d686417999f6d4d579
Sep 09 12:57:21.123297 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.123277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:21.127257 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:21.127232    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2360cec6_b8d9_443f_94d3_c93dc3768b70.slice/crio-97645f391a494903fc8ce37297d635a931cc5c31d781e1bc549bd18c23042627 WatchSource:0}: Error finding container 97645f391a494903fc8ce37297d635a931cc5c31d781e1bc549bd18c23042627: Status 404 returned error can't find the container with id 97645f391a494903fc8ce37297d635a931cc5c31d781e1bc549bd18c23042627
Sep 09 12:57:21.958343 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.958207    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" containerName="manager" containerID="cri-o://c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b" gracePeriod=2
Sep 09 12:57:21.960050 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.960018    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj" event={"ID":"d60e23d2-c572-4c93-ab81-59f477b7e4b2","Type":"ContainerStarted","Data":"fda010e72869937e4b2178b2d85ce6e74bd965d99a89820fb93a97b0aa0f43b5"}
Sep 09 12:57:21.960171 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.960089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj" event={"ID":"d60e23d2-c572-4c93-ab81-59f477b7e4b2","Type":"ContainerStarted","Data":"3108208ddea3b95aa7565ba82b481feb90fb040ba9dae1d686417999f6d4d579"}
Sep 09 12:57:21.960246 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.960166    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:21.960552 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:21.960522    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:21.962021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.961998    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" event={"ID":"2360cec6-b8d9-443f-94d3-c93dc3768b70","Type":"ContainerStarted","Data":"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"}
Sep 09 12:57:21.962333 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.962047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:21.962333 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.962071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" event={"ID":"2360cec6-b8d9-443f-94d3-c93dc3768b70","Type":"ContainerStarted","Data":"97645f391a494903fc8ce37297d635a931cc5c31d781e1bc549bd18c23042627"}
Sep 09 12:57:21.962564 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:21.962524    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:21.964019 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.963997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" event={"ID":"e13a1c01-feb5-4d3b-9b5f-36585128465f","Type":"ContainerStarted","Data":"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"}
Sep 09 12:57:21.964122 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.964037    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" event={"ID":"e13a1c01-feb5-4d3b-9b5f-36585128465f","Type":"ContainerStarted","Data":"25fffe76973f9284cfbb27c0ecd09015edb11cf899fb508d4ac150b7c1557211"}
Sep 09 12:57:21.964185 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.964127    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:21.980419 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.980379    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj" podStartSLOduration=1.980368013 podStartE2EDuration="1.980368013s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:21.978613252 +0000 UTC m=+540.013861162" watchObservedRunningTime="2026-09-09 12:57:21.980368013 +0000 UTC m=+540.015615919"
Sep 09 12:57:21.998436 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:21.998400    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" podStartSLOduration=1.9983911349999999 podStartE2EDuration="1.998391135s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:21.997852934 +0000 UTC m=+540.033100838" watchObservedRunningTime="2026-09-09 12:57:21.998391135 +0000 UTC m=+540.033639041"
Sep 09 12:57:22.023210 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.023167    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" podStartSLOduration=2.023154618 podStartE2EDuration="2.023154618s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:22.021181624 +0000 UTC m=+540.056429532" watchObservedRunningTime="2026-09-09 12:57:22.023154618 +0000 UTC m=+540.058402525"
Sep 09 12:57:22.024738 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.024715    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:22.199056 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.199032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:22.201183 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.201160    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:22.203076 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.203059    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:22.214519 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.214443    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume\") pod \"568a2d45-ffcc-4f88-b880-00db15b95298\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") "
Sep 09 12:57:22.214627 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.214533    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7npz6\" (UniqueName: \"kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6\") pod \"568a2d45-ffcc-4f88-b880-00db15b95298\" (UID: \"568a2d45-ffcc-4f88-b880-00db15b95298\") "
Sep 09 12:57:22.214894 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.214862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "568a2d45-ffcc-4f88-b880-00db15b95298" (UID: "568a2d45-ffcc-4f88-b880-00db15b95298"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:57:22.216774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.216746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6" (OuterVolumeSpecName: "kube-api-access-7npz6") pod "568a2d45-ffcc-4f88-b880-00db15b95298" (UID: "568a2d45-ffcc-4f88-b880-00db15b95298"). InnerVolumeSpecName "kube-api-access-7npz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:57:22.315999 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.315974    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/568a2d45-ffcc-4f88-b880-00db15b95298-extensions-socket-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:22.315999 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.315994    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7npz6\" (UniqueName: \"kubernetes.io/projected/568a2d45-ffcc-4f88-b880-00db15b95298-kube-api-access-7npz6\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:22.456571 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.456406    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-c9782\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="ee012789-6695-4d29-9764-effd4ef46c84" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-c9782"
Sep 09 12:57:22.458410 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.458379    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-67c5dcb48c-zqvvz\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:22.460354 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.460332    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32b18c1e-e425-40c4-b290-ad275a742ce6" path="/var/lib/kubelet/pods/32b18c1e-e425-40c4-b290-ad275a742ce6/volumes"
Sep 09 12:57:22.460684 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.460671    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="568a2d45-ffcc-4f88-b880-00db15b95298" path="/var/lib/kubelet/pods/568a2d45-ffcc-4f88-b880-00db15b95298/volumes"
Sep 09 12:57:22.460948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.460937    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee012789-6695-4d29-9764-effd4ef46c84" path="/var/lib/kubelet/pods/ee012789-6695-4d29-9764-effd4ef46c84/volumes"
Sep 09 12:57:22.971359 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.971329    2580 generic.go:356] "Generic (PLEG): container finished" podID="568a2d45-ffcc-4f88-b880-00db15b95298" containerID="c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b" exitCode=0
Sep 09 12:57:22.971819 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.971386    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-zqvvz"
Sep 09 12:57:22.971819 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.971462    2580 scope.go:117] "RemoveContainer" containerID="c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b"
Sep 09 12:57:22.980020 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.979991    2580 scope.go:117] "RemoveContainer" containerID="c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b"
Sep 09 12:57:22.980254 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:22.980233    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b\": container with ID starting with c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b not found: ID does not exist" containerID="c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b"
Sep 09 12:57:22.980309 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:22.980268    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b"} err="failed to get container status \"c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b\": rpc error: code = NotFound desc = could not find container \"c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b\": container with ID starting with c91f551d56f5db8d2c71cf97d050c9bfbd48fe44708a73b4b20d177052178f2b not found: ID does not exist"
Sep 09 12:57:24.629796 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:24.629756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:24.730455 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:24.730425    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:26.274107 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:26.274073    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:26.333809 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:26.333776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:31.290286 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:31.290246    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:31.389225 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:31.389183    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:32.973320 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:32.973289    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-4v6zj"
Sep 09 12:57:32.973765 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:32.973365    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:32.974218 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:32.974199    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:33.081484 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.081451    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:33.081710 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.081682    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" podUID="2360cec6-b8d9-443f-94d3-c93dc3768b70" containerName="manager" containerID="cri-o://a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285" gracePeriod=10
Sep 09 12:57:33.339246 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.339219    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:33.355005 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.354977    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 12:57:33.355353 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.355340    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2360cec6-b8d9-443f-94d3-c93dc3768b70" containerName="manager"
Sep 09 12:57:33.355402 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.355355    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2360cec6-b8d9-443f-94d3-c93dc3768b70" containerName="manager"
Sep 09 12:57:33.355434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.355426    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2360cec6-b8d9-443f-94d3-c93dc3768b70" containerName="manager"
Sep 09 12:57:33.357329 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.357314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.373412 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.373388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 12:57:33.396139 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.396114    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cd5f5\" (UniqueName: \"kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5\") pod \"2360cec6-b8d9-443f-94d3-c93dc3768b70\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") "
Sep 09 12:57:33.396259 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.396233    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume\") pod \"2360cec6-b8d9-443f-94d3-c93dc3768b70\" (UID: \"2360cec6-b8d9-443f-94d3-c93dc3768b70\") "
Sep 09 12:57:33.396347 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.396327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.396419 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.396367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h44p\" (UniqueName: \"kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.396638 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.396615    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2360cec6-b8d9-443f-94d3-c93dc3768b70" (UID: "2360cec6-b8d9-443f-94d3-c93dc3768b70"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:57:33.398671 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.398644    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5" (OuterVolumeSpecName: "kube-api-access-cd5f5") pod "2360cec6-b8d9-443f-94d3-c93dc3768b70" (UID: "2360cec6-b8d9-443f-94d3-c93dc3768b70"). InnerVolumeSpecName "kube-api-access-cd5f5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:57:33.497702 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.497637    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.497787 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.497769    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h44p\" (UniqueName: \"kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.497911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.497894    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2360cec6-b8d9-443f-94d3-c93dc3768b70-extensions-socket-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:33.497991 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.497917    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cd5f5\" (UniqueName: \"kubernetes.io/projected/2360cec6-b8d9-443f-94d3-c93dc3768b70-kube-api-access-cd5f5\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:33.497991 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.497952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.511563 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.511536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h44p\" (UniqueName: \"kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p\") pod \"kuadrant-operator-controller-manager-5769f99d9f-8dt26\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.669125 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.669097    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:33.947341 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:33.947318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 12:57:33.949755 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:57:33.949717    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467942ca_997c_478a_8b4e_1a7c7186cb41.slice/crio-a08f480983980d163b118471c7d1b88ca2bfe06a233228f4442606c94ab854c4 WatchSource:0}: Error finding container a08f480983980d163b118471c7d1b88ca2bfe06a233228f4442606c94ab854c4: Status 404 returned error can't find the container with id a08f480983980d163b118471c7d1b88ca2bfe06a233228f4442606c94ab854c4
Sep 09 12:57:34.038268 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.038238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" event={"ID":"467942ca-997c-478a-8b4e-1a7c7186cb41","Type":"ContainerStarted","Data":"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"}
Sep 09 12:57:34.038668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.038286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" event={"ID":"467942ca-997c-478a-8b4e-1a7c7186cb41","Type":"ContainerStarted","Data":"a08f480983980d163b118471c7d1b88ca2bfe06a233228f4442606c94ab854c4"}
Sep 09 12:57:34.038668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.038342    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:34.039744 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.039722    2580 generic.go:356] "Generic (PLEG): container finished" podID="2360cec6-b8d9-443f-94d3-c93dc3768b70" containerID="a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285" exitCode=0
Sep 09 12:57:34.039854 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.039755    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"
Sep 09 12:57:34.039854 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.039794    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" event={"ID":"2360cec6-b8d9-443f-94d3-c93dc3768b70","Type":"ContainerDied","Data":"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"}
Sep 09 12:57:34.039854 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.039840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl" event={"ID":"2360cec6-b8d9-443f-94d3-c93dc3768b70","Type":"ContainerDied","Data":"97645f391a494903fc8ce37297d635a931cc5c31d781e1bc549bd18c23042627"}
Sep 09 12:57:34.040012 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.039870    2580 scope.go:117] "RemoveContainer" containerID="a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"
Sep 09 12:57:34.049011 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.048995    2580 scope.go:117] "RemoveContainer" containerID="a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"
Sep 09 12:57:34.049258 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:34.049243    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285\": container with ID starting with a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285 not found: ID does not exist" containerID="a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"
Sep 09 12:57:34.049305 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.049266    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285"} err="failed to get container status \"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285\": rpc error: code = NotFound desc = could not find container \"a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285\": container with ID starting with a0854a59826ae660cedaff73aacc6966870640e4f4aab36dccd92ab9bce35285 not found: ID does not exist"
Sep 09 12:57:34.096993 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.096946    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" podStartSLOduration=1.096935655 podStartE2EDuration="1.096935655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:57:34.071490001 +0000 UTC m=+552.106737917" watchObservedRunningTime="2026-09-09 12:57:34.096935655 +0000 UTC m=+552.132183562"
Sep 09 12:57:34.098922 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.098906    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:34.118394 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.118372    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-fpzvl"]
Sep 09 12:57:34.452907 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:34.452875    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2360cec6-b8d9-443f-94d3-c93dc3768b70" path="/var/lib/kubelet/pods/2360cec6-b8d9-443f-94d3-c93dc3768b70/volumes"
Sep 09 12:57:36.155143 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:36.155106    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:36.219522 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:36.219475    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:43.001998 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:43.001959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:43.102219 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:43.102189    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:45.050223 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.050192    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 12:57:45.133532 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.133486    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:45.133709 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.133684    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" podUID="e13a1c01-feb5-4d3b-9b5f-36585128465f" containerName="manager" containerID="cri-o://35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5" gracePeriod=10
Sep 09 12:57:45.377133 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.377102    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:45.489524 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.489474    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume\") pod \"e13a1c01-feb5-4d3b-9b5f-36585128465f\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") "
Sep 09 12:57:45.489675 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.489577    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nskk4\" (UniqueName: \"kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4\") pod \"e13a1c01-feb5-4d3b-9b5f-36585128465f\" (UID: \"e13a1c01-feb5-4d3b-9b5f-36585128465f\") "
Sep 09 12:57:45.489849 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.489823    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e13a1c01-feb5-4d3b-9b5f-36585128465f" (UID: "e13a1c01-feb5-4d3b-9b5f-36585128465f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 12:57:45.491480 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.491457    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4" (OuterVolumeSpecName: "kube-api-access-nskk4") pod "e13a1c01-feb5-4d3b-9b5f-36585128465f" (UID: "e13a1c01-feb5-4d3b-9b5f-36585128465f"). InnerVolumeSpecName "kube-api-access-nskk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:57:45.590742 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.590716    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nskk4\" (UniqueName: \"kubernetes.io/projected/e13a1c01-feb5-4d3b-9b5f-36585128465f-kube-api-access-nskk4\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:45.590742 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:45.590738    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e13a1c01-feb5-4d3b-9b5f-36585128465f-extensions-socket-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:57:46.117332 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.117291    2580 generic.go:356] "Generic (PLEG): container finished" podID="e13a1c01-feb5-4d3b-9b5f-36585128465f" containerID="35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5" exitCode=0
Sep 09 12:57:46.117760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.117342    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"
Sep 09 12:57:46.117760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.117374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" event={"ID":"e13a1c01-feb5-4d3b-9b5f-36585128465f","Type":"ContainerDied","Data":"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"}
Sep 09 12:57:46.117760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.117446    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g" event={"ID":"e13a1c01-feb5-4d3b-9b5f-36585128465f","Type":"ContainerDied","Data":"25fffe76973f9284cfbb27c0ecd09015edb11cf899fb508d4ac150b7c1557211"}
Sep 09 12:57:46.117760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.117468    2580 scope.go:117] "RemoveContainer" containerID="35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"
Sep 09 12:57:46.127213 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.127198    2580 scope.go:117] "RemoveContainer" containerID="35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"
Sep 09 12:57:46.127481 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:57:46.127462    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5\": container with ID starting with 35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5 not found: ID does not exist" containerID="35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"
Sep 09 12:57:46.127585 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.127487    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5"} err="failed to get container status \"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5\": rpc error: code = NotFound desc = could not find container \"35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5\": container with ID starting with 35ad6c1c2f9973f07bd4ea7b0a41a9a3648b3f5232cc3c805252d3cc294b4de5 not found: ID does not exist"
Sep 09 12:57:46.172022 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.171999    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:46.178444 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.178422    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-67c5dcb48c-s442g"]
Sep 09 12:57:46.452658 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:46.452620    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e13a1c01-feb5-4d3b-9b5f-36585128465f" path="/var/lib/kubelet/pods/e13a1c01-feb5-4d3b-9b5f-36585128465f/volumes"
Sep 09 12:57:49.586696 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:49.586659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:57:49.988778 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:57:49.988744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:01.215228 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:01.215192    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:03.989974 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.989926    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:03.995828 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.995791    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:03.996262 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.996244    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e13a1c01-feb5-4d3b-9b5f-36585128465f" containerName="manager"
Sep 09 12:58:03.996365 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.996264    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e13a1c01-feb5-4d3b-9b5f-36585128465f" containerName="manager"
Sep 09 12:58:03.996365 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.996350    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e13a1c01-feb5-4d3b-9b5f-36585128465f" containerName="manager"
Sep 09 12:58:03.999016 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:03.998995    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.005934 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.005912    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 12:58:04.006048 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.005981    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kq5kw\""
Sep 09 12:58:04.012286 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.012266    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:04.095850 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.095815    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:04.129654 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.129622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.129808 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.129739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjbv\" (UniqueName: \"kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.230711 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.230674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjbv\" (UniqueName: \"kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.230914 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.230752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.231390 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.231364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.242092 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.242035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjbv\" (UniqueName: \"kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv\") pod \"limitador-limitador-8588749977-rgcw4\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") " pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.308896 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.308871    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:04.579924 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.579886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:04.581668 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:04.581642    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb02540b_19db_4131_b906_37c24382ea30.slice/crio-f2b4bb482337a51e8515c50eea7cd44fe73b60017fde056455879cd5c400fb88 WatchSource:0}: Error finding container f2b4bb482337a51e8515c50eea7cd44fe73b60017fde056455879cd5c400fb88: Status 404 returned error can't find the container with id f2b4bb482337a51e8515c50eea7cd44fe73b60017fde056455879cd5c400fb88
Sep 09 12:58:04.982565 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.982531    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:04.985631 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.985608    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:04.988434 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.988409    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-6kkhr\""
Sep 09 12:58:04.996585 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:04.996564    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:05.001357 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.001329    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:05.037543 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.037522    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gshtg\" (UniqueName: \"kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg\") pod \"authorino-7bcfd8f9cd-dbnzd\" (UID: \"d38b965f-538d-4b69-9e9d-de105e03d779\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:05.138456 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.138419    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gshtg\" (UniqueName: \"kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg\") pod \"authorino-7bcfd8f9cd-dbnzd\" (UID: \"d38b965f-538d-4b69-9e9d-de105e03d779\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:05.163640 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.163608    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:05.165790 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.165768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gshtg\" (UniqueName: \"kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg\") pod \"authorino-7bcfd8f9cd-dbnzd\" (UID: \"d38b965f-538d-4b69-9e9d-de105e03d779\") " pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:05.229032 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.229003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" event={"ID":"db02540b-19db-4131-b906-37c24382ea30","Type":"ContainerStarted","Data":"f2b4bb482337a51e8515c50eea7cd44fe73b60017fde056455879cd5c400fb88"}
Sep 09 12:58:05.240671 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.240609    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:05.242917 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.242900    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:05.254142 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.254122    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:05.296131 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.296105    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:05.340534 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.340480    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6k2h\" (UniqueName: \"kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h\") pod \"authorino-59c974bd48-9th29\" (UID: \"98b7c673-585d-47eb-b580-c231f934accd\") " pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:05.441526 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.441474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6k2h\" (UniqueName: \"kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h\") pod \"authorino-59c974bd48-9th29\" (UID: \"98b7c673-585d-47eb-b580-c231f934accd\") " pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:05.457355 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.457314    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6k2h\" (UniqueName: \"kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h\") pod \"authorino-59c974bd48-9th29\" (UID: \"98b7c673-585d-47eb-b580-c231f934accd\") " pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:05.553004 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.552943    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:05.575774 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.575751    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:05.711832 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:05.711791    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd38b965f_538d_4b69_9e9d_de105e03d779.slice/crio-b533d10a2a9698beca7d4c1ea230bd021f493aea240564b664463bf901920558 WatchSource:0}: Error finding container b533d10a2a9698beca7d4c1ea230bd021f493aea240564b664463bf901920558: Status 404 returned error can't find the container with id b533d10a2a9698beca7d4c1ea230bd021f493aea240564b664463bf901920558
Sep 09 12:58:05.983991 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:05.983968    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:05.985129 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:05.985105    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod98b7c673_585d_47eb_b580_c231f934accd.slice/crio-f1d11133010fe3852267dde61b00faf33b02e368953d34b1a2d9bc9da1b6c9a4 WatchSource:0}: Error finding container f1d11133010fe3852267dde61b00faf33b02e368953d34b1a2d9bc9da1b6c9a4: Status 404 returned error can't find the container with id f1d11133010fe3852267dde61b00faf33b02e368953d34b1a2d9bc9da1b6c9a4
Sep 09 12:58:06.236201 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:06.236129    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9th29" event={"ID":"98b7c673-585d-47eb-b580-c231f934accd","Type":"ContainerStarted","Data":"f1d11133010fe3852267dde61b00faf33b02e368953d34b1a2d9bc9da1b6c9a4"}
Sep 09 12:58:06.237253 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:06.237233    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" event={"ID":"d38b965f-538d-4b69-9e9d-de105e03d779","Type":"ContainerStarted","Data":"b533d10a2a9698beca7d4c1ea230bd021f493aea240564b664463bf901920558"}
Sep 09 12:58:06.238576 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:06.238558    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" event={"ID":"db02540b-19db-4131-b906-37c24382ea30","Type":"ContainerStarted","Data":"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"}
Sep 09 12:58:06.238724 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:06.238713    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:06.278891 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:06.278850    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" podStartSLOduration=2.120911037 podStartE2EDuration="3.278836716s" totalImagesPullingTime="1.157925679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:06.270952792 +0000 UTC m=+584.306200699" watchObservedRunningTime="2026-09-09 12:58:06.278836716 +0000 UTC m=+584.314084681"
Sep 09 12:58:08.252362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:08.252324    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9th29" event={"ID":"98b7c673-585d-47eb-b580-c231f934accd","Type":"ContainerStarted","Data":"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"}
Sep 09 12:58:08.253856 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:08.253832    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" event={"ID":"d38b965f-538d-4b69-9e9d-de105e03d779","Type":"ContainerStarted","Data":"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"}
Sep 09 12:58:08.308798 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:08.308749    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" podStartSLOduration=2.605540265 podStartE2EDuration="4.308733761s" totalImagesPullingTime="1.703193496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:08.307110238 +0000 UTC m=+586.342358147" watchObservedRunningTime="2026-09-09 12:58:08.308733761 +0000 UTC m=+586.343981704"
Sep 09 12:58:08.309103 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:08.309080    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-59c974bd48-9th29" podStartSLOduration=1.883094878 podStartE2EDuration="3.309074157s" totalImagesPullingTime="1.425979279s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:08.280530625 +0000 UTC m=+586.315778529" watchObservedRunningTime="2026-09-09 12:58:08.309074157 +0000 UTC m=+586.344322063"
Sep 09 12:58:08.326411 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:08.326381    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:10.268529 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:10.266963    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" podUID="d38b965f-538d-4b69-9e9d-de105e03d779" containerName="authorino" containerID="cri-o://ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6" gracePeriod=30
Sep 09 12:58:10.526954 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:10.526897    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:10.584622 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:10.584596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gshtg\" (UniqueName: \"kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg\") pod \"d38b965f-538d-4b69-9e9d-de105e03d779\" (UID: \"d38b965f-538d-4b69-9e9d-de105e03d779\") "
Sep 09 12:58:10.586487 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:10.586454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg" (OuterVolumeSpecName: "kube-api-access-gshtg") pod "d38b965f-538d-4b69-9e9d-de105e03d779" (UID: "d38b965f-538d-4b69-9e9d-de105e03d779"). InnerVolumeSpecName "kube-api-access-gshtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:10.686125 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:10.686102    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gshtg\" (UniqueName: \"kubernetes.io/projected/d38b965f-538d-4b69-9e9d-de105e03d779-kube-api-access-gshtg\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:11.274217 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.274179    2580 generic.go:356] "Generic (PLEG): container finished" podID="d38b965f-538d-4b69-9e9d-de105e03d779" containerID="ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6" exitCode=0
Sep 09 12:58:11.274642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.274225    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd"
Sep 09 12:58:11.274642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.274245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" event={"ID":"d38b965f-538d-4b69-9e9d-de105e03d779","Type":"ContainerDied","Data":"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"}
Sep 09 12:58:11.274642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.274283    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-dbnzd" event={"ID":"d38b965f-538d-4b69-9e9d-de105e03d779","Type":"ContainerDied","Data":"b533d10a2a9698beca7d4c1ea230bd021f493aea240564b664463bf901920558"}
Sep 09 12:58:11.274642 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.274304    2580 scope.go:117] "RemoveContainer" containerID="ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"
Sep 09 12:58:11.283445 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.283425    2580 scope.go:117] "RemoveContainer" containerID="ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"
Sep 09 12:58:11.283751 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:58:11.283734    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6\": container with ID starting with ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6 not found: ID does not exist" containerID="ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"
Sep 09 12:58:11.283824 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.283761    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6"} err="failed to get container status \"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6\": rpc error: code = NotFound desc = could not find container \"ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6\": container with ID starting with ac370da8f94933e2624c18ed3f2ed0afe23159653af6401488cfedf243b3c5a6 not found: ID does not exist"
Sep 09 12:58:11.298650 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.298627    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:11.301405 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:11.301385    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-dbnzd"]
Sep 09 12:58:12.306179 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:12.306144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:12.453487 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:12.453458    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d38b965f-538d-4b69-9e9d-de105e03d779" path="/var/lib/kubelet/pods/d38b965f-538d-4b69-9e9d-de105e03d779/volumes"
Sep 09 12:58:17.245969 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:17.245941    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:19.172019 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.171978    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:19.172464 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.172195    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" podUID="db02540b-19db-4131-b906-37c24382ea30" containerName="limitador" containerID="cri-o://4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9" gracePeriod=30
Sep 09 12:58:19.717930 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.717903    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:19.856722 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.856646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hdjbv\" (UniqueName: \"kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv\") pod \"db02540b-19db-4131-b906-37c24382ea30\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") "
Sep 09 12:58:19.856860 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.856737    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file\") pod \"db02540b-19db-4131-b906-37c24382ea30\" (UID: \"db02540b-19db-4131-b906-37c24382ea30\") "
Sep 09 12:58:19.857087 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.857054    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file" (OuterVolumeSpecName: "config-file") pod "db02540b-19db-4131-b906-37c24382ea30" (UID: "db02540b-19db-4131-b906-37c24382ea30"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 12:58:19.858652 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.858628    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv" (OuterVolumeSpecName: "kube-api-access-hdjbv") pod "db02540b-19db-4131-b906-37c24382ea30" (UID: "db02540b-19db-4131-b906-37c24382ea30"). InnerVolumeSpecName "kube-api-access-hdjbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:19.957795 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.957774    2580 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/db02540b-19db-4131-b906-37c24382ea30-config-file\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:19.957795 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:19.957795    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hdjbv\" (UniqueName: \"kubernetes.io/projected/db02540b-19db-4131-b906-37c24382ea30-kube-api-access-hdjbv\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:20.329867 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.329832    2580 generic.go:356] "Generic (PLEG): container finished" podID="db02540b-19db-4131-b906-37c24382ea30" containerID="4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9" exitCode=0
Sep 09 12:58:20.330342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.329872    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4"
Sep 09 12:58:20.330342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.329916    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" event={"ID":"db02540b-19db-4131-b906-37c24382ea30","Type":"ContainerDied","Data":"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"}
Sep 09 12:58:20.330342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.329975    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-rgcw4" event={"ID":"db02540b-19db-4131-b906-37c24382ea30","Type":"ContainerDied","Data":"f2b4bb482337a51e8515c50eea7cd44fe73b60017fde056455879cd5c400fb88"}
Sep 09 12:58:20.330342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.330010    2580 scope.go:117] "RemoveContainer" containerID="4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"
Sep 09 12:58:20.339860 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.339843    2580 scope.go:117] "RemoveContainer" containerID="4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"
Sep 09 12:58:20.340125 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:58:20.340107    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9\": container with ID starting with 4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9 not found: ID does not exist" containerID="4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"
Sep 09 12:58:20.340173 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.340134    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9"} err="failed to get container status \"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9\": rpc error: code = NotFound desc = could not find container \"4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9\": container with ID starting with 4e1494ff83e3018d0e153c4de1f4090c5daa3ef0fb99dc7f84923a752dace5a9 not found: ID does not exist"
Sep 09 12:58:20.352702 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.352676    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:20.358980 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.358953    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-rgcw4"]
Sep 09 12:58:20.452284 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:20.452261    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="db02540b-19db-4131-b906-37c24382ea30" path="/var/lib/kubelet/pods/db02540b-19db-4131-b906-37c24382ea30/volumes"
Sep 09 12:58:22.410837 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:22.410811    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:58:22.413917 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:22.413896    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 12:58:23.429965 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.429925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:23.533129 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.533095    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:23.921977 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.921945    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-594dz"]
Sep 09 12:58:23.922525 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922494    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d38b965f-538d-4b69-9e9d-de105e03d779" containerName="authorino"
Sep 09 12:58:23.922577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922529    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d38b965f-538d-4b69-9e9d-de105e03d779" containerName="authorino"
Sep 09 12:58:23.922577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922568    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="db02540b-19db-4131-b906-37c24382ea30" containerName="limitador"
Sep 09 12:58:23.922648 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922577    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="db02540b-19db-4131-b906-37c24382ea30" containerName="limitador"
Sep 09 12:58:23.922682 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922672    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="db02540b-19db-4131-b906-37c24382ea30" containerName="limitador"
Sep 09 12:58:23.922718 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.922683    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d38b965f-538d-4b69-9e9d-de105e03d779" containerName="authorino"
Sep 09 12:58:23.925905 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.925885    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:23.928640 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.928614    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 12:58:23.928760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.928687    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-h46tn\""
Sep 09 12:58:23.928760 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.928718    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 12:58:23.929462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.929443    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 12:58:23.934197 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.934177    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-594dz"]
Sep 09 12:58:23.959605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:23.959576    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:24.089010 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.088972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e6ce55eb-699a-4893-ac5e-348344c63fc7-data\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.089010 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.089013    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhv7p\" (UniqueName: \"kubernetes.io/projected/e6ce55eb-699a-4893-ac5e-348344c63fc7-kube-api-access-hhv7p\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.189800 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.189729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e6ce55eb-699a-4893-ac5e-348344c63fc7-data\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.189800 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.189767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhv7p\" (UniqueName: \"kubernetes.io/projected/e6ce55eb-699a-4893-ac5e-348344c63fc7-kube-api-access-hhv7p\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.190102 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.190082    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e6ce55eb-699a-4893-ac5e-348344c63fc7-data\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.206871 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.206848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhv7p\" (UniqueName: \"kubernetes.io/projected/e6ce55eb-699a-4893-ac5e-348344c63fc7-kube-api-access-hhv7p\") pod \"postgres-69fb4594cf-594dz\" (UID: \"e6ce55eb-699a-4893-ac5e-348344c63fc7\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.238668 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.238647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:24.512060 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:24.512035    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-594dz"]
Sep 09 12:58:24.513433 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:24.513409    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6ce55eb_699a_4893_ac5e_348344c63fc7.slice/crio-207e73dbec8ea380b8565f84474651d6f9343082b4b59cd01438902e6c9829e2 WatchSource:0}: Error finding container 207e73dbec8ea380b8565f84474651d6f9343082b4b59cd01438902e6c9829e2: Status 404 returned error can't find the container with id 207e73dbec8ea380b8565f84474651d6f9343082b4b59cd01438902e6c9829e2
Sep 09 12:58:25.359487 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:25.359452    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz" event={"ID":"e6ce55eb-699a-4893-ac5e-348344c63fc7","Type":"ContainerStarted","Data":"207e73dbec8ea380b8565f84474651d6f9343082b4b59cd01438902e6c9829e2"}
Sep 09 12:58:30.400612 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:30.400578    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz" event={"ID":"e6ce55eb-699a-4893-ac5e-348344c63fc7","Type":"ContainerStarted","Data":"cfe35c441d862512bb8723702a2aebc5b8c9041f282a4d958587670224575e7d"}
Sep 09 12:58:30.400972 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:30.400650    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:30.421717 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:30.421668    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz" podStartSLOduration=2.041307491 podStartE2EDuration="7.421655419s" totalImagesPullingTime="5.380347928s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:30.419645152 +0000 UTC m=+608.454893060" watchObservedRunningTime="2026-09-09 12:58:30.421655419 +0000 UTC m=+608.456903362"
Sep 09 12:58:36.434626 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.434595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-594dz"
Sep 09 12:58:36.750023 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.749941    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:36.851551 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.851493    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:36.989160 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.989127    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-9gc9r"]
Sep 09 12:58:36.992931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.992907    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:36.998859 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:36.998830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gc9r"]
Sep 09 12:58:37.044491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.044429    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gc9r"]
Sep 09 12:58:37.044696 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:58:37.044677    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lbk9p], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-557f876f69-9gc9r" podUID="57fa4277-1b09-4a90-85d5-b6682239315b"
Sep 09 12:58:37.068676 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.068648    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-77lmp"]
Sep 09 12:58:37.072115 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.072101    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.074342 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.074323    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 09 12:58:37.080015 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.079971    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-77lmp"]
Sep 09 12:58:37.097763 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.097743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbk9p\" (UniqueName: \"kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p\") pod \"authorino-557f876f69-9gc9r\" (UID: \"57fa4277-1b09-4a90-85d5-b6682239315b\") " pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:37.198697 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.198664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbk9p\" (UniqueName: \"kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p\") pod \"authorino-557f876f69-9gc9r\" (UID: \"57fa4277-1b09-4a90-85d5-b6682239315b\") " pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:37.198857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.198708    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vdkz\" (UniqueName: \"kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.198857 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.198772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.206707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.206675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbk9p\" (UniqueName: \"kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p\") pod \"authorino-557f876f69-9gc9r\" (UID: \"57fa4277-1b09-4a90-85d5-b6682239315b\") " pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:37.239680 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.239652    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-77lmp"]
Sep 09 12:58:37.239885 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:58:37.239862    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-2vdkz tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-77lmp" podUID="60d733d8-216d-4940-aa08-5a190a14e9a6"
Sep 09 12:58:37.272113 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.272086    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 12:58:37.275458 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.275442    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.284406 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.284383    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 12:58:37.299669 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.299618    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.299758 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.299694    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vdkz\" (UniqueName: \"kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.302093 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.302066    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.307251 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.307231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vdkz\" (UniqueName: \"kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz\") pod \"authorino-6d4dfc649f-77lmp\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") " pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.400702 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.400653    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.400849 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.400718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ftphd\" (UniqueName: \"kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.436309 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.436285    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.436660 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.436286    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:37.443403 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.443388    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:37.446376 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.446358    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:37.501803 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.501776    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.501935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.501816    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ftphd\" (UniqueName: \"kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.504292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.504268    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.510785 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.510763    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ftphd\" (UniqueName: \"kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd\") pod \"authorino-58b7dd8954-b9f9m\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") " pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.584848 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.584798    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 12:58:37.602705 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.602686    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert\") pod \"60d733d8-216d-4940-aa08-5a190a14e9a6\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") "
Sep 09 12:58:37.602793 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.602763    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbk9p\" (UniqueName: \"kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p\") pod \"57fa4277-1b09-4a90-85d5-b6682239315b\" (UID: \"57fa4277-1b09-4a90-85d5-b6682239315b\") "
Sep 09 12:58:37.602839 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.602804    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vdkz\" (UniqueName: \"kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz\") pod \"60d733d8-216d-4940-aa08-5a190a14e9a6\" (UID: \"60d733d8-216d-4940-aa08-5a190a14e9a6\") "
Sep 09 12:58:37.604777 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.604746    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "60d733d8-216d-4940-aa08-5a190a14e9a6" (UID: "60d733d8-216d-4940-aa08-5a190a14e9a6"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:58:37.604777 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.604751    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz" (OuterVolumeSpecName: "kube-api-access-2vdkz") pod "60d733d8-216d-4940-aa08-5a190a14e9a6" (UID: "60d733d8-216d-4940-aa08-5a190a14e9a6"). InnerVolumeSpecName "kube-api-access-2vdkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:37.604902 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.604800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p" (OuterVolumeSpecName: "kube-api-access-lbk9p") pod "57fa4277-1b09-4a90-85d5-b6682239315b" (UID: "57fa4277-1b09-4a90-85d5-b6682239315b"). InnerVolumeSpecName "kube-api-access-lbk9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:37.703951 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.703917    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/60d733d8-216d-4940-aa08-5a190a14e9a6-tls-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:37.703951 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.703947    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbk9p\" (UniqueName: \"kubernetes.io/projected/57fa4277-1b09-4a90-85d5-b6682239315b-kube-api-access-lbk9p\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:37.704111 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.703960    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vdkz\" (UniqueName: \"kubernetes.io/projected/60d733d8-216d-4940-aa08-5a190a14e9a6-kube-api-access-2vdkz\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:37.855061 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:37.855033    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 12:58:37.856921 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:37.856885    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode583eba7_170e_4566_b0e3_fcbdaedce60b.slice/crio-b016b84082521808f5eb7d765ba1c80bd50e8599429a18feca00d718796b815a WatchSource:0}: Error finding container b016b84082521808f5eb7d765ba1c80bd50e8599429a18feca00d718796b815a: Status 404 returned error can't find the container with id b016b84082521808f5eb7d765ba1c80bd50e8599429a18feca00d718796b815a
Sep 09 12:58:38.443964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.443886    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gc9r"
Sep 09 12:58:38.443964 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.443920    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" event={"ID":"e583eba7-170e-4566-b0e3-fcbdaedce60b","Type":"ContainerStarted","Data":"b016b84082521808f5eb7d765ba1c80bd50e8599429a18feca00d718796b815a"}
Sep 09 12:58:38.444329 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.444108    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-77lmp"
Sep 09 12:58:38.480956 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.480835    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-77lmp"]
Sep 09 12:58:38.484441 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.484414    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-77lmp"]
Sep 09 12:58:38.512615 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.512577    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gc9r"]
Sep 09 12:58:38.515491 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:38.515464    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gc9r"]
Sep 09 12:58:39.451552 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.451497    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" event={"ID":"e583eba7-170e-4566-b0e3-fcbdaedce60b","Type":"ContainerStarted","Data":"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"}
Sep 09 12:58:39.465606 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.465559    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" podStartSLOduration=1.936634267 podStartE2EDuration="2.465544759s" totalImagesPullingTime="528.910492ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:39.464211429 +0000 UTC m=+617.499459337" watchObservedRunningTime="2026-09-09 12:58:39.465544759 +0000 UTC m=+617.500792665"
Sep 09 12:58:39.491953 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.491918    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:39.492154 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.492125    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-59c974bd48-9th29" podUID="98b7c673-585d-47eb-b580-c231f934accd" containerName="authorino" containerID="cri-o://ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7" gracePeriod=30
Sep 09 12:58:39.731787 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.731766    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:39.922362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.922314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w6k2h\" (UniqueName: \"kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h\") pod \"98b7c673-585d-47eb-b580-c231f934accd\" (UID: \"98b7c673-585d-47eb-b580-c231f934accd\") "
Sep 09 12:58:39.924357 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:39.924333    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h" (OuterVolumeSpecName: "kube-api-access-w6k2h") pod "98b7c673-585d-47eb-b580-c231f934accd" (UID: "98b7c673-585d-47eb-b580-c231f934accd"). InnerVolumeSpecName "kube-api-access-w6k2h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:40.023251 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.023184    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w6k2h\" (UniqueName: \"kubernetes.io/projected/98b7c673-585d-47eb-b580-c231f934accd-kube-api-access-w6k2h\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:40.452911 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.452875    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57fa4277-1b09-4a90-85d5-b6682239315b" path="/var/lib/kubelet/pods/57fa4277-1b09-4a90-85d5-b6682239315b/volumes"
Sep 09 12:58:40.453295 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.453153    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60d733d8-216d-4940-aa08-5a190a14e9a6" path="/var/lib/kubelet/pods/60d733d8-216d-4940-aa08-5a190a14e9a6/volumes"
Sep 09 12:58:40.457411 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.457388    2580 generic.go:356] "Generic (PLEG): container finished" podID="98b7c673-585d-47eb-b580-c231f934accd" containerID="ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7" exitCode=0
Sep 09 12:58:40.457577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.457435    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-59c974bd48-9th29"
Sep 09 12:58:40.457577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.457458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9th29" event={"ID":"98b7c673-585d-47eb-b580-c231f934accd","Type":"ContainerDied","Data":"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"}
Sep 09 12:58:40.457577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.457532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-59c974bd48-9th29" event={"ID":"98b7c673-585d-47eb-b580-c231f934accd","Type":"ContainerDied","Data":"f1d11133010fe3852267dde61b00faf33b02e368953d34b1a2d9bc9da1b6c9a4"}
Sep 09 12:58:40.457577 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.457556    2580 scope.go:117] "RemoveContainer" containerID="ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"
Sep 09 12:58:40.468474 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.468458    2580 scope.go:117] "RemoveContainer" containerID="ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"
Sep 09 12:58:40.468751 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:58:40.468730    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7\": container with ID starting with ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7 not found: ID does not exist" containerID="ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"
Sep 09 12:58:40.468824 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.468761    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7"} err="failed to get container status \"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7\": rpc error: code = NotFound desc = could not find container \"ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7\": container with ID starting with ede526f3aff91d3eaad75b0760eaf25aeabc8faba55740b136ede23120d768b7 not found: ID does not exist"
Sep 09 12:58:40.477021 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.476997    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:40.482280 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:40.482259    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-59c974bd48-9th29"]
Sep 09 12:58:42.452320 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:42.452288    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98b7c673-585d-47eb-b580-c231f934accd" path="/var/lib/kubelet/pods/98b7c673-585d-47eb-b580-c231f934accd/volumes"
Sep 09 12:58:44.638708 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.638670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:44.663042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.663013    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:58:44.663448 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.663431    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98b7c673-585d-47eb-b580-c231f934accd" containerName="authorino"
Sep 09 12:58:44.663561 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.663450    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98b7c673-585d-47eb-b580-c231f934accd" containerName="authorino"
Sep 09 12:58:44.663629 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.663575    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98b7c673-585d-47eb-b580-c231f934accd" containerName="authorino"
Sep 09 12:58:44.668979 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.668961    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.671115 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.671095    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 12:58:44.671115 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.671109    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g2wjn\""
Sep 09 12:58:44.671234 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.671194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 12:58:44.671932 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.671914    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 12:58:44.677790 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.677772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:58:44.755108 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.755085    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dw9r\" (UniqueName: \"kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.755238 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.755131    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.755238 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.755173    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.856473 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.856403    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.856639 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.856493    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dw9r\" (UniqueName: \"kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.856639 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.856586    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.858953 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.858930    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.859046 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.858951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.861777 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.861756    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 12:58:44.865773 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.865756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:44.874939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.874914    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dw9r\" (UniqueName: \"kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r\") pod \"maas-controller-588455fbb-s9q9t\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") " pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:44.878207 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.878184    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 12:58:44.957187 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.957157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:44.957325 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.957221    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:44.957325 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.957246    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9q8r\" (UniqueName: \"kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:44.981050 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:44.981025    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:45.058380 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.058350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.058565 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.058391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9q8r\" (UniqueName: \"kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.058565 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.058448    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.061564 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.061481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.062009 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.061746    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.066893 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.066869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9q8r\" (UniqueName: \"kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r\") pod \"maas-controller-885499b4f-xq4z5\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") " pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.193765 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.193741    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:45.256439 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.256307    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:58:45.259307 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:45.259272    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8172ab43_ebf2_4d28_a868_c34bd4654ed0.slice/crio-0ba6faa784c121b22b51a067bcce4cd41fafef7e4029eb1d229cb8b550be0eb9 WatchSource:0}: Error finding container 0ba6faa784c121b22b51a067bcce4cd41fafef7e4029eb1d229cb8b550be0eb9: Status 404 returned error can't find the container with id 0ba6faa784c121b22b51a067bcce4cd41fafef7e4029eb1d229cb8b550be0eb9
Sep 09 12:58:45.465359 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.465333    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 12:58:45.466949 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:58:45.466931    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8feb91b2_4adb_442b_8333_8da3c2329680.slice/crio-032309c78cd8090c73e1f82dc20ade2f50dabcfe65c04a8b324a31c60717afc7 WatchSource:0}: Error finding container 032309c78cd8090c73e1f82dc20ade2f50dabcfe65c04a8b324a31c60717afc7: Status 404 returned error can't find the container with id 032309c78cd8090c73e1f82dc20ade2f50dabcfe65c04a8b324a31c60717afc7
Sep 09 12:58:45.485536 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.485491    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-xq4z5" event={"ID":"8feb91b2-4adb-442b-8333-8da3c2329680","Type":"ContainerStarted","Data":"032309c78cd8090c73e1f82dc20ade2f50dabcfe65c04a8b324a31c60717afc7"}
Sep 09 12:58:45.486692 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:45.486672    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588455fbb-s9q9t" event={"ID":"8172ab43-ebf2-4d28-a868-c34bd4654ed0","Type":"ContainerStarted","Data":"0ba6faa784c121b22b51a067bcce4cd41fafef7e4029eb1d229cb8b550be0eb9"}
Sep 09 12:58:46.735605 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:46.735556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:48.115956 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.115923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:48.213931 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.213834    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:48.514636 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.514540    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588455fbb-s9q9t" event={"ID":"8172ab43-ebf2-4d28-a868-c34bd4654ed0","Type":"ContainerStarted","Data":"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"}
Sep 09 12:58:48.514799 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.514758    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:48.516831 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.516809    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-xq4z5" event={"ID":"8feb91b2-4adb-442b-8333-8da3c2329680","Type":"ContainerStarted","Data":"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"}
Sep 09 12:58:48.516940 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.516869    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:48.534707 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.534661    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-588455fbb-s9q9t" podStartSLOduration=1.84801797 podStartE2EDuration="4.534646553s" totalImagesPullingTime="2.686628583s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:48.532867858 +0000 UTC m=+626.568115781" watchObservedRunningTime="2026-09-09 12:58:48.534646553 +0000 UTC m=+626.569894459"
Sep 09 12:58:48.551255 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:48.551209    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-885499b4f-xq4z5" podStartSLOduration=2.083969851 podStartE2EDuration="4.55118955s" totalImagesPullingTime="2.467219699s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:58:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:58:48.5508065 +0000 UTC m=+626.586054408" watchObservedRunningTime="2026-09-09 12:58:48.55118955 +0000 UTC m=+626.586437457"
Sep 09 12:58:59.526826 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.526791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 12:58:59.527232 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.526974    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:59.588528 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.588483    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:58:59.588770 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.588743    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-588455fbb-s9q9t" podUID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" containerName="manager" containerID="cri-o://5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e" gracePeriod=30
Sep 09 12:58:59.658991 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.658964    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:58:59.829034 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.829009    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:58:59.879820 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.879790    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert\") pod \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") "
Sep 09 12:58:59.879975 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.879860    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dw9r\" (UniqueName: \"kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r\") pod \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") "
Sep 09 12:58:59.879975 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.879900    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs\") pod \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\" (UID: \"8172ab43-ebf2-4d28-a868-c34bd4654ed0\") "
Sep 09 12:58:59.881987 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.881959    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8172ab43-ebf2-4d28-a868-c34bd4654ed0" (UID: "8172ab43-ebf2-4d28-a868-c34bd4654ed0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:58:59.882112 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.882037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8172ab43-ebf2-4d28-a868-c34bd4654ed0" (UID: "8172ab43-ebf2-4d28-a868-c34bd4654ed0"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 12:58:59.882112 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.882039    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r" (OuterVolumeSpecName: "kube-api-access-9dw9r") pod "8172ab43-ebf2-4d28-a868-c34bd4654ed0" (UID: "8172ab43-ebf2-4d28-a868-c34bd4654ed0"). InnerVolumeSpecName "kube-api-access-9dw9r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 12:58:59.980935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.980901    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-webhook-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:59.980935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.980932    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dw9r\" (UniqueName: \"kubernetes.io/projected/8172ab43-ebf2-4d28-a868-c34bd4654ed0-kube-api-access-9dw9r\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:58:59.980935 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:58:59.980943    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8172ab43-ebf2-4d28-a868-c34bd4654ed0-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 12:59:00.587226 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.587193    2580 generic.go:356] "Generic (PLEG): container finished" podID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" containerID="5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e" exitCode=0
Sep 09 12:59:00.587687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.587234    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588455fbb-s9q9t" event={"ID":"8172ab43-ebf2-4d28-a868-c34bd4654ed0","Type":"ContainerDied","Data":"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"}
Sep 09 12:59:00.587687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.587250    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-588455fbb-s9q9t"
Sep 09 12:59:00.587687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.587286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-588455fbb-s9q9t" event={"ID":"8172ab43-ebf2-4d28-a868-c34bd4654ed0","Type":"ContainerDied","Data":"0ba6faa784c121b22b51a067bcce4cd41fafef7e4029eb1d229cb8b550be0eb9"}
Sep 09 12:59:00.587687 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.587310    2580 scope.go:117] "RemoveContainer" containerID="5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"
Sep 09 12:59:00.596781 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.596623    2580 scope.go:117] "RemoveContainer" containerID="5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"
Sep 09 12:59:00.596898 ip-10-0-137-1 kubenswrapper[2580]: E0909 12:59:00.596880    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e\": container with ID starting with 5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e not found: ID does not exist" containerID="5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"
Sep 09 12:59:00.596939 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.596905    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e"} err="failed to get container status \"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e\": rpc error: code = NotFound desc = could not find container \"5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e\": container with ID starting with 5dd482f7f2727ef43041763d5f0b156ad04e60eec2bef35974c1fff057e3255e not found: ID does not exist"
Sep 09 12:59:00.603474 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.603451    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:59:00.606883 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:00.606863    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-588455fbb-s9q9t"]
Sep 09 12:59:02.453461 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:02.453427    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" path="/var/lib/kubelet/pods/8172ab43-ebf2-4d28-a868-c34bd4654ed0/volumes"
Sep 09 12:59:16.008340 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:16.008305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:16.064830 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:16.064802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:19.792323 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:19.792284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:19.896012 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:19.895973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.312397 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.312348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.412549 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.412514    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.467015 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.466984    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 12:59:21.467362 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.467350    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" containerName="manager"
Sep 09 12:59:21.467415 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.467363    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" containerName="manager"
Sep 09 12:59:21.467454 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.467418    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8172ab43-ebf2-4d28-a868-c34bd4654ed0" containerName="manager"
Sep 09 12:59:21.472453 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.472436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.475141 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.475122    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-dbjk4\""
Sep 09 12:59:21.475947 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.475926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 12:59:21.476106 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.475959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 12:59:21.479837 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.479785    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 12:59:21.492788 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.492755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.551137 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.551110    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.551137 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.551137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.551292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.551166    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.551292 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.551223    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zs2q\" (UniqueName: \"kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.620766 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.620694    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.652033 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.652012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zs2q\" (UniqueName: \"kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.652157 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.652075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.652157 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.652094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.652157 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.652127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.652948 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.652925    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.655335 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.655258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.655451 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.655405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.660694 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.660664    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zs2q\" (UniqueName: \"kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q\") pod \"maas-api-5dc88f4c96-78ft6\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") " pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.786584 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.786556    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:21.819606 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.819521    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.929124 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.929081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:21.958147 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.958117    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"]
Sep 09 12:59:21.962726 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.962701    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:21.965295 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.965273    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 12:59:21.973042 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:21.973019    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"]
Sep 09 12:59:22.057141 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.057096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.057433 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.057414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8z2rz\" (UniqueName: \"kubernetes.io/projected/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-kube-api-access-8z2rz\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.074121 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.074095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 12:59:22.077329 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:59:22.077284    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b8042c8_4b1e_4035_9c4e_619bf105b659.slice/crio-bea89aaa117d582fb57b253247e62b697e1f211205c066b2eafcc4041f129cf8 WatchSource:0}: Error finding container bea89aaa117d582fb57b253247e62b697e1f211205c066b2eafcc4041f129cf8: Status 404 returned error can't find the container with id bea89aaa117d582fb57b253247e62b697e1f211205c066b2eafcc4041f129cf8
Sep 09 12:59:22.079091 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.079069    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 12:59:22.159081 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.159047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.159081 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.159080    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8z2rz\" (UniqueName: \"kubernetes.io/projected/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-kube-api-access-8z2rz\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.159752 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.159731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.167082 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.167057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8z2rz\" (UniqueName: \"kubernetes.io/projected/adca4535-86ea-4c97-8b3e-1e1636b4ab0d-kube-api-access-8z2rz\") pod \"payload-processing-5b54b7fbb7-wq5fl\" (UID: \"adca4535-86ea-4c97-8b3e-1e1636b4ab0d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.275897 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.275813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:22.550028 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.549990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"]
Sep 09 12:59:22.551123 ip-10-0-137-1 kubenswrapper[2580]: W0909 12:59:22.551102    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadca4535_86ea_4c97_8b3e_1e1636b4ab0d.slice/crio-3e6d86559e7e58d7f6b2ce620dafd0a9ac9556a3fe714ed0ec05a0e02738cbd6 WatchSource:0}: Error finding container 3e6d86559e7e58d7f6b2ce620dafd0a9ac9556a3fe714ed0ec05a0e02738cbd6: Status 404 returned error can't find the container with id 3e6d86559e7e58d7f6b2ce620dafd0a9ac9556a3fe714ed0ec05a0e02738cbd6
Sep 09 12:59:22.717289 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.717248    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl" event={"ID":"adca4535-86ea-4c97-8b3e-1e1636b4ab0d","Type":"ContainerStarted","Data":"3e6d86559e7e58d7f6b2ce620dafd0a9ac9556a3fe714ed0ec05a0e02738cbd6"}
Sep 09 12:59:22.718843 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:22.718815    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" event={"ID":"6b8042c8-4b1e-4035-9c4e-619bf105b659","Type":"ContainerStarted","Data":"bea89aaa117d582fb57b253247e62b697e1f211205c066b2eafcc4041f129cf8"}
Sep 09 12:59:24.734397 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:24.734305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" event={"ID":"6b8042c8-4b1e-4035-9c4e-619bf105b659","Type":"ContainerStarted","Data":"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"}
Sep 09 12:59:24.734836 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:24.734466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:24.735915 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:24.735897    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl" event={"ID":"adca4535-86ea-4c97-8b3e-1e1636b4ab0d","Type":"ContainerStarted","Data":"60d0cca3464ec35bc6599094482714227a9afea537a37c7bc5bc0953d6020634"}
Sep 09 12:59:24.736043 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:24.736028    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:24.752072 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:24.752029    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" podStartSLOduration=1.81986495 podStartE2EDuration="3.752016265s" totalImagesPullingTime="1.932151315s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:59:24.749261612 +0000 UTC m=+662.784509518" watchObservedRunningTime="2026-09-09 12:59:24.752016265 +0000 UTC m=+662.787264220"
Sep 09 12:59:30.747462 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:30.747431    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 12:59:30.765853 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:30.765807    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl" podStartSLOduration=7.901650626 podStartE2EDuration="9.765792447s" totalImagesPullingTime="1.864141821s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 12:59:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 12:59:24.763883473 +0000 UTC m=+662.799131377" watchObservedRunningTime="2026-09-09 12:59:30.765792447 +0000 UTC m=+668.801040354"
Sep 09 12:59:31.727573 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:31.727533    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:31.829251 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:31.829219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 12:59:35.744081 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:35.744047    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-wq5fl"
Sep 09 12:59:55.397012 ip-10-0-137-1 kubenswrapper[2580]: I0909 12:59:55.396977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:06.906061 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:06.905967    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:20.670051 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.669761    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"]
Sep 09 13:00:20.673658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.673637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.675902 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.675879    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 13:00:20.676015 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.675886    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-pjccl\""
Sep 09 13:00:20.676727 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.676708    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 13:00:20.676832 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.676756    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 09 13:00:20.683224 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.683206    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"]
Sep 09 13:00:20.757812 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.757777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.757974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.757833    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.757974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.757890    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.757974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.757944    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.757974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.757973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmf29\" (UniqueName: \"kubernetes.io/projected/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-kube-api-access-zmf29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.758135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.758051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.803496 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.803454    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:20.858658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858628    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.858828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858671    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.858828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858690    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.858828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmf29\" (UniqueName: \"kubernetes.io/projected/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-kube-api-access-zmf29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.858828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.858828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.858773    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.859524 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.859392    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.859524 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.859455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.859754 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.859593    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.862542 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.862080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.862542 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.862318    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.873666 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.873640    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmf29\" (UniqueName: \"kubernetes.io/projected/61cae3b7-30fb-433e-8d49-3ec4e1e97c3c-kube-api-access-zmf29\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv\" (UID: \"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:20.987907 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:20.987823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:21.260915 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:21.260890    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"]
Sep 09 13:00:21.262994 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:00:21.262967    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61cae3b7_30fb_433e_8d49_3ec4e1e97c3c.slice/crio-38c02c836fe4943aafdea9650a10a506e59b3e6d1fda7cbbeb2773a92f303e45 WatchSource:0}: Error finding container 38c02c836fe4943aafdea9650a10a506e59b3e6d1fda7cbbeb2773a92f303e45: Status 404 returned error can't find the container with id 38c02c836fe4943aafdea9650a10a506e59b3e6d1fda7cbbeb2773a92f303e45
Sep 09 13:00:22.070287 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:22.070250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv" event={"ID":"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c","Type":"ContainerStarted","Data":"38c02c836fe4943aafdea9650a10a506e59b3e6d1fda7cbbeb2773a92f303e45"}
Sep 09 13:00:27.103307 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:27.103273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv" event={"ID":"61cae3b7-30fb-433e-8d49-3ec4e1e97c3c","Type":"ContainerStarted","Data":"96a1ea417983edd5d37b13d1c43bb208e41d67d85bd9609ebec6dca563835a39"}
Sep 09 13:00:27.120392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:27.120345    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv" podStartSLOduration=1.890477079 podStartE2EDuration="7.120329292s" totalImagesPullingTime="5.229852213s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:00:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:00:27.11844208 +0000 UTC m=+725.153689988" watchObservedRunningTime="2026-09-09 13:00:27.120329292 +0000 UTC m=+725.155577200"
Sep 09 13:00:29.401106 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:29.401068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:30.988870 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:30.988825    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:30.989280 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:30.988899    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:31.001715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:31.001687    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:31.139855 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:31.139826    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv"
Sep 09 13:00:36.868926 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.868881    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"]
Sep 09 13:00:36.873620 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.873597    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.875989 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.875966    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 09 13:00:36.882893 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.882869    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"]
Sep 09 13:00:36.896560 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896536    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.896673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896565    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.896673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.896764 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896683    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whmrt\" (UniqueName: \"kubernetes.io/projected/62a4c5d5-627c-433e-9b47-653a51707824-kube-api-access-whmrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.896764 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.896877 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.896808    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62a4c5d5-627c-433e-9b47-653a51707824-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.968390 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.968354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:36.997639 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.997823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62a4c5d5-627c-433e-9b47-653a51707824-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.997823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.997823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997705    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.997823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997729    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.997823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.997800    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whmrt\" (UniqueName: \"kubernetes.io/projected/62a4c5d5-627c-433e-9b47-653a51707824-kube-api-access-whmrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.998101 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.998079    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.998161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.998122    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:36.998244 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:36.998222    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:37.000024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:37.000005    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/62a4c5d5-627c-433e-9b47-653a51707824-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:37.000267 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:37.000251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/62a4c5d5-627c-433e-9b47-653a51707824-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:37.009595 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:37.009566    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whmrt\" (UniqueName: \"kubernetes.io/projected/62a4c5d5-627c-433e-9b47-653a51707824-kube-api-access-whmrt\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm\" (UID: \"62a4c5d5-627c-433e-9b47-653a51707824\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:37.184106 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:37.184054    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:37.457676 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:37.457647    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"]
Sep 09 13:00:38.169640 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:38.169604    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm" event={"ID":"62a4c5d5-627c-433e-9b47-653a51707824","Type":"ContainerStarted","Data":"2eaecf66fe3756f3a513355298b6e553df405de53c3f5694847912ff92e9fc06"}
Sep 09 13:00:38.170081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:38.169664    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm" event={"ID":"62a4c5d5-627c-433e-9b47-653a51707824","Type":"ContainerStarted","Data":"41c228dc04d7c4a16d8b6de7373f36759860698c082312f6243c4fd589f72d54"}
Sep 09 13:00:38.188985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:38.188930    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm" podStartSLOduration=1.820971636 podStartE2EDuration="2.188914599s" totalImagesPullingTime="367.942963ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:00:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:00:38.187455346 +0000 UTC m=+736.222703254" watchObservedRunningTime="2026-09-09 13:00:38.188914599 +0000 UTC m=+736.224162506"
Sep 09 13:00:40.934112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:40.934071    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:40.991542 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:40.991486    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:42.091532 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:42.091484    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:00:47.184436 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:47.184394    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:47.184909 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:47.184459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:47.196246 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:47.196221    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:47.236382 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:47.236360    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm"
Sep 09 13:00:59.288805 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:00:59.288770    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:01:03.014605 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:01:03.014566    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:12.796112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:12.796078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:12.849152 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:12.849118    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:39.917316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:39.917278    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:42.970353 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:42.970317    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56f8459dc5-m7glz"]
Sep 09 13:02:42.973955 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:42.973935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:42.988491 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:42.988470    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56f8459dc5-m7glz"]
Sep 09 13:02:43.074642 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.074612    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-tls-cert\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.074803 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.074660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z87ns\" (UniqueName: \"kubernetes.io/projected/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-kube-api-access-z87ns\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.175315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.175285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-tls-cert\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.175495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.175333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z87ns\" (UniqueName: \"kubernetes.io/projected/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-kube-api-access-z87ns\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.177750 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.177723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-tls-cert\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.182882 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.182854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z87ns\" (UniqueName: \"kubernetes.io/projected/daa75492-94a7-46f2-b9e5-fa8df4e3c09d-kube-api-access-z87ns\") pod \"authorino-56f8459dc5-m7glz\" (UID: \"daa75492-94a7-46f2-b9e5-fa8df4e3c09d\") " pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.282997 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.282911    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56f8459dc5-m7glz"
Sep 09 13:02:43.556966 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.556940    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56f8459dc5-m7glz"]
Sep 09 13:02:43.558575 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:02:43.558546    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddaa75492_94a7_46f2_b9e5_fa8df4e3c09d.slice/crio-d264b1838601c09ca31f74eb7a9e34dcb54685293c761145b86d4af2f772a162 WatchSource:0}: Error finding container d264b1838601c09ca31f74eb7a9e34dcb54685293c761145b86d4af2f772a162: Status 404 returned error can't find the container with id d264b1838601c09ca31f74eb7a9e34dcb54685293c761145b86d4af2f772a162
Sep 09 13:02:43.888465 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:43.888370    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f8459dc5-m7glz" event={"ID":"daa75492-94a7-46f2-b9e5-fa8df4e3c09d","Type":"ContainerStarted","Data":"d264b1838601c09ca31f74eb7a9e34dcb54685293c761145b86d4af2f772a162"}
Sep 09 13:02:44.898114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:44.898077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56f8459dc5-m7glz" event={"ID":"daa75492-94a7-46f2-b9e5-fa8df4e3c09d","Type":"ContainerStarted","Data":"52e353f5cae133dadefe8ced343d49d4632d684953d70d0cf4f71ac9cf6ebc57"}
Sep 09 13:02:44.917960 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:44.917899    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56f8459dc5-m7glz" podStartSLOduration=2.273262392 podStartE2EDuration="2.917878343s" totalImagesPullingTime="644.615951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:02:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:02:44.913980564 +0000 UTC m=+862.949228471" watchObservedRunningTime="2026-09-09 13:02:44.917878343 +0000 UTC m=+862.953126253"
Sep 09 13:02:44.944979 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:44.944947    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 13:02:44.945167 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:44.945125    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" podUID="e583eba7-170e-4566-b0e3-fcbdaedce60b" containerName="authorino" containerID="cri-o://659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18" gracePeriod=30
Sep 09 13:02:45.224981 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.224951    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 13:02:45.296781 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.296748    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ftphd\" (UniqueName: \"kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd\") pod \"e583eba7-170e-4566-b0e3-fcbdaedce60b\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") "
Sep 09 13:02:45.296949 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.296821    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert\") pod \"e583eba7-170e-4566-b0e3-fcbdaedce60b\" (UID: \"e583eba7-170e-4566-b0e3-fcbdaedce60b\") "
Sep 09 13:02:45.298917 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.298870    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd" (OuterVolumeSpecName: "kube-api-access-ftphd") pod "e583eba7-170e-4566-b0e3-fcbdaedce60b" (UID: "e583eba7-170e-4566-b0e3-fcbdaedce60b"). InnerVolumeSpecName "kube-api-access-ftphd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:02:45.307406 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.307382    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "e583eba7-170e-4566-b0e3-fcbdaedce60b" (UID: "e583eba7-170e-4566-b0e3-fcbdaedce60b"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:02:45.398208 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.398168    2580 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/e583eba7-170e-4566-b0e3-fcbdaedce60b-tls-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:02:45.398208 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.398202    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ftphd\" (UniqueName: \"kubernetes.io/projected/e583eba7-170e-4566-b0e3-fcbdaedce60b-kube-api-access-ftphd\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:02:45.544320 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.544226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:45.678539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.678482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:45.906914 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.906875    2580 generic.go:356] "Generic (PLEG): container finished" podID="e583eba7-170e-4566-b0e3-fcbdaedce60b" containerID="659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18" exitCode=0
Sep 09 13:02:45.906914 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.906912    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58b7dd8954-b9f9m"
Sep 09 13:02:45.907568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.906967    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" event={"ID":"e583eba7-170e-4566-b0e3-fcbdaedce60b","Type":"ContainerDied","Data":"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"}
Sep 09 13:02:45.907568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.907011    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58b7dd8954-b9f9m" event={"ID":"e583eba7-170e-4566-b0e3-fcbdaedce60b","Type":"ContainerDied","Data":"b016b84082521808f5eb7d765ba1c80bd50e8599429a18feca00d718796b815a"}
Sep 09 13:02:45.907568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.907032    2580 scope.go:117] "RemoveContainer" containerID="659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"
Sep 09 13:02:45.920478 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.920457    2580 scope.go:117] "RemoveContainer" containerID="659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"
Sep 09 13:02:45.920820 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:02:45.920801    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18\": container with ID starting with 659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18 not found: ID does not exist" containerID="659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"
Sep 09 13:02:45.920903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.920832    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18"} err="failed to get container status \"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18\": rpc error: code = NotFound desc = could not find container \"659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18\": container with ID starting with 659c2deb606b4456a40bab6f2fc4c9328b8403cd1c355d0f4fa5dfdcf103da18 not found: ID does not exist"
Sep 09 13:02:45.932950 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.932928    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 13:02:45.942692 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:45.942671    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58b7dd8954-b9f9m"]
Sep 09 13:02:46.453032 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:46.452991    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e583eba7-170e-4566-b0e3-fcbdaedce60b" path="/var/lib/kubelet/pods/e583eba7-170e-4566-b0e3-fcbdaedce60b/volumes"
Sep 09 13:02:46.853698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:46.853602    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:02:46.943632 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:02:46.943581    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:05.544657 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:05.544625    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:06.186191 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:06.186152    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:06.246013 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:06.245978    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:07.083937 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.083904    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:07.209782 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.209738    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:07.609416 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.609386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:07.625062 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.625036    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7855-140702222177840"]
Sep 09 13:03:07.625637 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.625621    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e583eba7-170e-4566-b0e3-fcbdaedce60b" containerName="authorino"
Sep 09 13:03:07.625688 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.625640    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e583eba7-170e-4566-b0e3-fcbdaedce60b" containerName="authorino"
Sep 09 13:03:07.625752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.625741    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e583eba7-170e-4566-b0e3-fcbdaedce60b" containerName="authorino"
Sep 09 13:03:07.629260 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.629242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:07.631857 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.631837    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s2rqh\""
Sep 09 13:03:07.634110 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.634085    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7855-140702222177840"]
Sep 09 13:03:07.790830 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.790785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbhbw\" (UniqueName: \"kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw\") pod \"test-curl-7855-140702222177840\" (UID: \"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d\") " pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:07.892340 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.892310    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbhbw\" (UniqueName: \"kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw\") pod \"test-curl-7855-140702222177840\" (UID: \"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d\") " pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:07.899940 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.899912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbhbw\" (UniqueName: \"kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw\") pod \"test-curl-7855-140702222177840\" (UID: \"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d\") " pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:07.940032 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:07.940005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:08.213399 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:08.213370    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7855-140702222177840"]
Sep 09 13:03:08.215277 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:08.215247    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode02e2ab0_72a2_4aa3_8488_0d6087c5ee1d.slice/crio-49b0ed6aa09d027927fbe6c795f843b246ae85e9e633f2de9b5f644048de2790 WatchSource:0}: Error finding container 49b0ed6aa09d027927fbe6c795f843b246ae85e9e633f2de9b5f644048de2790: Status 404 returned error can't find the container with id 49b0ed6aa09d027927fbe6c795f843b246ae85e9e633f2de9b5f644048de2790
Sep 09 13:03:09.053140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:09.053117    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7855-140702222177840" event={"ID":"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d","Type":"ContainerStarted","Data":"49b0ed6aa09d027927fbe6c795f843b246ae85e9e633f2de9b5f644048de2790"}
Sep 09 13:03:09.162755 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:09.162712    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:09.263433 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:09.263395    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:10.060379 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:10.060337    2580 generic.go:356] "Generic (PLEG): container finished" podID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" containerID="dc74b85e853823093cc9e248637c7207d56c51bd75c3313629d9b44ef8f04f07" exitCode=0
Sep 09 13:03:10.060607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:10.060428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7855-140702222177840" event={"ID":"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d","Type":"ContainerDied","Data":"dc74b85e853823093cc9e248637c7207d56c51bd75c3313629d9b44ef8f04f07"}
Sep 09 13:03:10.595233 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:10.595200    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:10.696194 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:10.696150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:10.755703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:10.755644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:11.073215 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.072728    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:11.152222 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.152184    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:11.215475 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.215437    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:11.234659 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.234637    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7855-140702222177840_e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d/test-curl-7855-140702222177840/0.log"
Sep 09 13:03:11.255764 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.255737    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7855-140702222177840"]
Sep 09 13:03:11.257975 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.257957    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7855-140702222177840"]
Sep 09 13:03:11.325789 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.325710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mbhbw\" (UniqueName: \"kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw\") pod \"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d\" (UID: \"e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d\") "
Sep 09 13:03:11.327806 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.327779    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw" (OuterVolumeSpecName: "kube-api-access-mbhbw") pod "e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" (UID: "e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d"). InnerVolumeSpecName "kube-api-access-mbhbw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:11.373565 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.373538    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7855-140702222178096"]
Sep 09 13:03:11.373966 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.373954    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" containerName="test-curl-7855-140702222177840"
Sep 09 13:03:11.374021 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.373968    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" containerName="test-curl-7855-140702222177840"
Sep 09 13:03:11.374070 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.374061    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" containerName="test-curl-7855-140702222177840"
Sep 09 13:03:11.377203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.377188    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:11.382563 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.382540    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7855-140702222178096"]
Sep 09 13:03:11.426736 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.426707    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbhbw\" (UniqueName: \"kubernetes.io/projected/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d-kube-api-access-mbhbw\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:03:11.527276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.527236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmbhp\" (UniqueName: \"kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp\") pod \"test-curl-7855-140702222178096\" (UID: \"309256db-0157-4636-bb08-053d31a3b016\") " pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:11.628646 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.628561    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmbhp\" (UniqueName: \"kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp\") pod \"test-curl-7855-140702222178096\" (UID: \"309256db-0157-4636-bb08-053d31a3b016\") " pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:11.636490 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.636461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmbhp\" (UniqueName: \"kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp\") pod \"test-curl-7855-140702222178096\" (UID: \"309256db-0157-4636-bb08-053d31a3b016\") " pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:11.687932 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.687906    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:11.960077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:11.960053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7855-140702222178096"]
Sep 09 13:03:11.961550 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:11.961525    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod309256db_0157_4636_bb08_053d31a3b016.slice/crio-96e3dd36810310c28192331f30823040b9f71e596a3e27f86341e37c825d9c40 WatchSource:0}: Error finding container 96e3dd36810310c28192331f30823040b9f71e596a3e27f86341e37c825d9c40: Status 404 returned error can't find the container with id 96e3dd36810310c28192331f30823040b9f71e596a3e27f86341e37c825d9c40
Sep 09 13:03:12.078286 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:12.078250    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7855-140702222178096" event={"ID":"309256db-0157-4636-bb08-053d31a3b016","Type":"ContainerStarted","Data":"96e3dd36810310c28192331f30823040b9f71e596a3e27f86341e37c825d9c40"}
Sep 09 13:03:12.079637 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:12.079618    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49b0ed6aa09d027927fbe6c795f843b246ae85e9e633f2de9b5f644048de2790"
Sep 09 13:03:12.079637 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:12.079637    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:12.089928 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:03:12.089903    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7855-140702222177840\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:12.451027 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:03:12.450989    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7855-140702222177840\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" pod="opendatahub/test-curl-7855-140702222177840"
Sep 09 13:03:12.453237 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:12.453217    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d" path="/var/lib/kubelet/pods/e02e2ab0-72a2-4aa3-8488-0d6087c5ee1d/volumes"
Sep 09 13:03:13.087820 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:13.087729    2580 generic.go:356] "Generic (PLEG): container finished" podID="309256db-0157-4636-bb08-053d31a3b016" containerID="f23de2b105a064fbf4d163a4c95608e25b8b4748f0d185abef0387ec0491e9be" exitCode=0
Sep 09 13:03:13.088187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:13.087811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7855-140702222178096" event={"ID":"309256db-0157-4636-bb08-053d31a3b016","Type":"ContainerDied","Data":"f23de2b105a064fbf4d163a4c95608e25b8b4748f0d185abef0387ec0491e9be"}
Sep 09 13:03:14.228187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.228165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:14.245924 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.245896    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7855-140702222178096_309256db-0157-4636-bb08-053d31a3b016/test-curl-7855-140702222178096/0.log"
Sep 09 13:03:14.266032 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.265988    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7855-140702222178096"]
Sep 09 13:03:14.268309 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.268288    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7855-140702222178096"]
Sep 09 13:03:14.352821 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.352755    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lmbhp\" (UniqueName: \"kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp\") pod \"309256db-0157-4636-bb08-053d31a3b016\" (UID: \"309256db-0157-4636-bb08-053d31a3b016\") "
Sep 09 13:03:14.354786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.354761    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp" (OuterVolumeSpecName: "kube-api-access-lmbhp") pod "309256db-0157-4636-bb08-053d31a3b016" (UID: "309256db-0157-4636-bb08-053d31a3b016"). InnerVolumeSpecName "kube-api-access-lmbhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:14.452918 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.452888    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="309256db-0157-4636-bb08-053d31a3b016" path="/var/lib/kubelet/pods/309256db-0157-4636-bb08-053d31a3b016/volumes"
Sep 09 13:03:14.453636 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:14.453617    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lmbhp\" (UniqueName: \"kubernetes.io/projected/309256db-0157-4636-bb08-053d31a3b016-kube-api-access-lmbhp\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:03:15.100937 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.100856    2580 scope.go:117] "RemoveContainer" containerID="f23de2b105a064fbf4d163a4c95608e25b8b4748f0d185abef0387ec0491e9be"
Sep 09 13:03:15.100937 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.100856    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7855-140702222178096"
Sep 09 13:03:15.140114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.140087    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:03:15.140732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.140702    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="309256db-0157-4636-bb08-053d31a3b016" containerName="test-curl-7855-140702222178096"
Sep 09 13:03:15.140871 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.140739    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="309256db-0157-4636-bb08-053d31a3b016" containerName="test-curl-7855-140702222178096"
Sep 09 13:03:15.140950 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.140868    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="309256db-0157-4636-bb08-053d31a3b016" containerName="test-curl-7855-140702222178096"
Sep 09 13:03:15.145275 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.145257    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.147489 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.147466    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 13:03:15.147489 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.147478    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-b4ea80-openshift-default-dockercfg-tv2bj\""
Sep 09 13:03:15.158759 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.158737    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:03:15.260067 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260035    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260132    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260183    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260312    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjlkz\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260347    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.260419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.260381    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.360861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.360771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.360861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.360840    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361095 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.360873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361095 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.360896    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjlkz\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361095 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361016    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361095 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361156    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361276    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361290    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361598 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361384    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.361598 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361494    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.362009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.361985    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.363178 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.363148    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.363575 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.363556    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.368224 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.368196    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjlkz\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.368365 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.368348    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token\") pod \"e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") " pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.457912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.457879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:15.737338 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.737307    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:03:15.737786 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:15.737756    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod062a7568_3680_4eae_9718_678cf1ae344b.slice/crio-42b9b7aefee938ddb4505a7656583e957ffed76c85149f56fa9e943ebfab3d36 WatchSource:0}: Error finding container 42b9b7aefee938ddb4505a7656583e957ffed76c85149f56fa9e943ebfab3d36: Status 404 returned error can't find the container with id 42b9b7aefee938ddb4505a7656583e957ffed76c85149f56fa9e943ebfab3d36
Sep 09 13:03:15.774391 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:15.774357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:16.112424 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:16.112138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerStarted","Data":"42b9b7aefee938ddb4505a7656583e957ffed76c85149f56fa9e943ebfab3d36"}
Sep 09 13:03:18.376340 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.376295    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:18.381418 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.381396    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.384223 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.384200    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-d0fb1d64\""
Sep 09 13:03:18.384335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.384233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-d0fb1d64-dockercfg-qf9sj\""
Sep 09 13:03:18.391486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.391460    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:18.447872 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.447832    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:18.491900 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.491865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.492062 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.492039    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkgsr\" (UniqueName: \"kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.548112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.548069    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:18.593368 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.593340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkgsr\" (UniqueName: \"kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.593523 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.593390    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.594125 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.594105    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.600869 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.600847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkgsr\" (UniqueName: \"kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr\") pod \"payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.693554 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.693522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:18.967938 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:18.967911    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:18.968489 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:18.968467    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32bafd3b_7bde_4908_85ae_c6498651d2b9.slice/crio-b39059e20ccde25d595e9a216e6db8985237e0d87d5f66d12fd7c43922167545 WatchSource:0}: Error finding container b39059e20ccde25d595e9a216e6db8985237e0d87d5f66d12fd7c43922167545: Status 404 returned error can't find the container with id b39059e20ccde25d595e9a216e6db8985237e0d87d5f66d12fd7c43922167545
Sep 09 13:03:19.137574 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:19.137286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" event={"ID":"32bafd3b-7bde-4908-85ae-c6498651d2b9","Type":"ContainerStarted","Data":"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"}
Sep 09 13:03:19.137574 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:19.137402    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" event={"ID":"32bafd3b-7bde-4908-85ae-c6498651d2b9","Type":"ContainerStarted","Data":"b39059e20ccde25d595e9a216e6db8985237e0d87d5f66d12fd7c43922167545"}
Sep 09 13:03:19.137950 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:19.137876    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:19.154975 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:19.154925    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" podStartSLOduration=1.154911115 podStartE2EDuration="1.154911115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:19.154134982 +0000 UTC m=+897.189382890" watchObservedRunningTime="2026-09-09 13:03:19.154911115 +0000 UTC m=+897.190159022"
Sep 09 13:03:22.466161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:22.466132    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:03:22.468845 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:22.468819    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:03:24.266931 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.266892    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:24.267217 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.266964    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:24.267217 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.266992    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:24.317700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.317654    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:24.473569 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.473464    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:24.568877 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.568787    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:24.668169 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:24.668136    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:25.179793 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:25.179758    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerStarted","Data":"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"}
Sep 09 13:03:25.199020 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:25.198972    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podStartSLOduration=1.6721587329999998 podStartE2EDuration="10.198953658s" totalImagesPullingTime="8.526794925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:25.197547874 +0000 UTC m=+903.232795781" watchObservedRunningTime="2026-09-09 13:03:25.198953658 +0000 UTC m=+903.234201565"
Sep 09 13:03:25.458809 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:25.458723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:25.464112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:25.464076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:26.184962 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:26.184931    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:26.185856 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:26.185835    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:26.818270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:26.818229    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:26.925529 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:26.925477    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:27.054537 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:27.054469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:27.150247 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:27.150214    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:27.286858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:27.286826    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:27.384073 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:27.384040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:28.293903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.293820    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:03:28.298566 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.298535    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.301321 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.301295    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-b4ea80-serving-cert\""
Sep 09 13:03:28.314530 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.311070    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:03:28.341858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.341829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:28.384389 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.384356    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6hgmf\" (UniqueName: \"kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.384607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.384414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.384607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.384438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.384607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.384473    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.457667 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.457631    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:28.486007 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.485973    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6hgmf\" (UniqueName: \"kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.486187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.486035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.486187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.486062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.486187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.486086    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.486362 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:03:28.486185    2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-serving-cert: secret "maas-api-e2e-autores-b4ea80-serving-cert" not found
Sep 09 13:03:28.486362 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:03:28.486259    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls podName:9082e4b4-8b7f-4d00-bbc2-48bb9885c088 nodeName:}" failed. No retries permitted until 2026-09-09 13:03:28.986238556 +0000 UTC m=+907.021486453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls") pod "maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" (UID: "9082e4b4-8b7f-4d00-bbc2-48bb9885c088") : secret "maas-api-e2e-autores-b4ea80-serving-cert" not found
Sep 09 13:03:28.486757 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.486735    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.488711 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.488690    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.493808 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.493787    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6hgmf\" (UniqueName: \"kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.558981 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.558888    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:28.905266 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.905238    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:28.990935 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.990903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:28.993551 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:28.993496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") pod \"maas-api-e2e-autores-b4ea80-64f88c444-6l4xt\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:29.006081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.006050    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:29.045454 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.045421    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:03:29.049528 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.049498    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.051760 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.051740    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-b4ea80-dockercfg-c4stf\""
Sep 09 13:03:29.051872 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.051778    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-b4ea80\""
Sep 09 13:03:29.058822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.058625    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:03:29.092018 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.091992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.092103 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.092019    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvx4\" (UniqueName: \"kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.105725 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.105700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:29.193273 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.193198    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.193273 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.193244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvx4\" (UniqueName: \"kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.194040 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.194022    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.200976 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.200953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvx4\" (UniqueName: \"kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4\") pod \"payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") " pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.218016 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.217970    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:29.362834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.362794    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:29.495308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.495272    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:03:29.497632 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:29.497607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9082e4b4_8b7f_4d00_bbc2_48bb9885c088.slice/crio-0f056e6d433314940e180314a914b5d312a3faa68e6c5b058d936155833739ea WatchSource:0}: Error finding container 0f056e6d433314940e180314a914b5d312a3faa68e6c5b058d936155833739ea: Status 404 returned error can't find the container with id 0f056e6d433314940e180314a914b5d312a3faa68e6c5b058d936155833739ea
Sep 09 13:03:29.636751 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:29.636720    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:03:29.637834 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:29.637802    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7030446_4937_4535_8c01_3a8396c4a342.slice/crio-dff449c30916d875ef1487f926c3449faf5d713a2d9fc1d2da4025b93a20a137 WatchSource:0}: Error finding container dff449c30916d875ef1487f926c3449faf5d713a2d9fc1d2da4025b93a20a137: Status 404 returned error can't find the container with id dff449c30916d875ef1487f926c3449faf5d713a2d9fc1d2da4025b93a20a137
Sep 09 13:03:30.145380 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.145350    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:30.217423 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.217325    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" event={"ID":"9082e4b4-8b7f-4d00-bbc2-48bb9885c088","Type":"ContainerStarted","Data":"e62758cad47f5e7746a81924869fb66b33b165a14ed351fc8d7a4b4fb4dd83cf"}
Sep 09 13:03:30.217423 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.217410    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:30.217691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.217441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" event={"ID":"9082e4b4-8b7f-4d00-bbc2-48bb9885c088","Type":"ContainerStarted","Data":"0f056e6d433314940e180314a914b5d312a3faa68e6c5b058d936155833739ea"}
Sep 09 13:03:30.219522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.219406    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" event={"ID":"b7030446-4937-4535-8c01-3a8396c4a342","Type":"ContainerStarted","Data":"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"}
Sep 09 13:03:30.219522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.219458    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" event={"ID":"b7030446-4937-4535-8c01-3a8396c4a342","Type":"ContainerStarted","Data":"dff449c30916d875ef1487f926c3449faf5d713a2d9fc1d2da4025b93a20a137"}
Sep 09 13:03:30.219522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.219489    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:30.238250 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.238195    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" podStartSLOduration=1.775370953 podStartE2EDuration="2.238174406s" totalImagesPullingTime="462.803453ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:30.233704945 +0000 UTC m=+908.268952847" watchObservedRunningTime="2026-09-09 13:03:30.238174406 +0000 UTC m=+908.273422315"
Sep 09 13:03:30.252331 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:30.252284    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" podStartSLOduration=1.252266576 podStartE2EDuration="1.252266576s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:30.248446455 +0000 UTC m=+908.283694362" watchObservedRunningTime="2026-09-09 13:03:30.252266576 +0000 UTC m=+908.287514484"
Sep 09 13:03:31.479993 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:31.479949    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:33.376704 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:33.376661    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:33.481926 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:33.481894    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:33.577190 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:33.577131    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:33.749834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:33.749797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:34.072760 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:34.072671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:36.242087 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:36.242057    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:03:37.812414 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:37.812368    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:37.914398 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:37.914359    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:38.663734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:38.663696    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:38.760876 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:38.760844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:39.890795 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:39.890759    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:39.990782 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:39.990753    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:40.694888 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:40.694849    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:40.794395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:40.794349    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:41.238179 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:41.237873    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:03:41.978335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:41.978296    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:03:41.983842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:41.983819    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:41.986562 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:41.986537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-7c991b-serving-cert\""
Sep 09 13:03:41.993198 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:41.993175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:03:42.027465 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.027404    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:42.091203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.091162    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:42.116911 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.116878    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vb4t9\" (UniqueName: \"kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.117063 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.116933    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.117063 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.117020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.117156 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.117130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.190427 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.190388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:42.218299 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.217695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vb4t9\" (UniqueName: \"kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.218299 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.217757    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.218299 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.217803    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.218299 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.217870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.218707 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.218612    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.220875 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.220844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.221113 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.221090    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.226948 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.226919    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vb4t9\" (UniqueName: \"kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9\") pod \"maas-api-e2e-inf-b-7c991b-d66495686-5k4pp\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.299528 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.299436    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:42.509694 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.509656    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:42.595784 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.595747    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:03:42.596336 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.596308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:42.598462 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:03:42.598438    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod963e57b9_4908_4be3_aa79_7c1225647ff3.slice/crio-a5f1ff6d7b3a5d687a96d0a9f2ee533986850861594fc11779170be11bed4701 WatchSource:0}: Error finding container a5f1ff6d7b3a5d687a96d0a9f2ee533986850861594fc11779170be11bed4701: Status 404 returned error can't find the container with id a5f1ff6d7b3a5d687a96d0a9f2ee533986850861594fc11779170be11bed4701
Sep 09 13:03:42.907023 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:42.906987    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:43.318650 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:43.318556    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" event={"ID":"963e57b9-4908-4be3-aa79-7c1225647ff3","Type":"ContainerStarted","Data":"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"}
Sep 09 13:03:43.318650 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:43.318627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" event={"ID":"963e57b9-4908-4be3-aa79-7c1225647ff3","Type":"ContainerStarted","Data":"a5f1ff6d7b3a5d687a96d0a9f2ee533986850861594fc11779170be11bed4701"}
Sep 09 13:03:43.319071 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:43.318753    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:43.336014 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:43.335970    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" podStartSLOduration=1.9582753990000001 podStartE2EDuration="2.335955297s" totalImagesPullingTime="377.679898ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:03:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:03:43.334181231 +0000 UTC m=+921.369429137" watchObservedRunningTime="2026-09-09 13:03:43.335955297 +0000 UTC m=+921.371203204"
Sep 09 13:03:43.800946 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:43.800907    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:45.333394 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.333361    2580 generic.go:356] "Generic (PLEG): container finished" podID="062a7568-3680-4eae-9718-678cf1ae344b" containerID="5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865" exitCode=0
Sep 09 13:03:45.333832 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.333436    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerDied","Data":"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"}
Sep 09 13:03:45.333875 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.333836    2580 scope.go:117] "RemoveContainer" containerID="5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"
Sep 09 13:03:45.335802 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.335772    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:45.335921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.335840    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:45.335921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:45.335871    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:03:46.343837 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:46.343804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerStarted","Data":"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"}
Sep 09 13:03:46.458822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:46.458791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:46.459077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:46.458947    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:46.459206 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:46.459107    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:47.458580 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:47.458549    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:47.458968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:47.458618    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:47.798056 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:47.797961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:48.458829 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.458797    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:48.459221 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.458859    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:48.484732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.484703    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:48.485228 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.485199    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" podUID="32bafd3b-7bde-4908-85ae-c6498651d2b9" containerName="payload-pre-processing" containerID="cri-o://55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c" gracePeriod=30
Sep 09 13:03:48.803525 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.803483    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:48.880989 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.880959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume\") pod \"32bafd3b-7bde-4908-85ae-c6498651d2b9\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") "
Sep 09 13:03:48.881147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.881056    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkgsr\" (UniqueName: \"kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr\") pod \"32bafd3b-7bde-4908-85ae-c6498651d2b9\" (UID: \"32bafd3b-7bde-4908-85ae-c6498651d2b9\") "
Sep 09 13:03:48.881339 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.881311    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32bafd3b-7bde-4908-85ae-c6498651d2b9" (UID: "32bafd3b-7bde-4908-85ae-c6498651d2b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:03:48.882998 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.882980    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr" (OuterVolumeSpecName: "kube-api-access-gkgsr") pod "32bafd3b-7bde-4908-85ae-c6498651d2b9" (UID: "32bafd3b-7bde-4908-85ae-c6498651d2b9"). InnerVolumeSpecName "kube-api-access-gkgsr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:03:48.982673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.982644    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32bafd3b-7bde-4908-85ae-c6498651d2b9-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:03:48.982673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:48.982671    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkgsr\" (UniqueName: \"kubernetes.io/projected/32bafd3b-7bde-4908-85ae-c6498651d2b9-kube-api-access-gkgsr\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:03:49.158736 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.158692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:49.335046 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.334318    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:03:49.364328 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.364295    2580 generic.go:356] "Generic (PLEG): container finished" podID="32bafd3b-7bde-4908-85ae-c6498651d2b9" containerID="55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c" exitCode=0
Sep 09 13:03:49.364593 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.364344    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"
Sep 09 13:03:49.364744 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.364364    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" event={"ID":"32bafd3b-7bde-4908-85ae-c6498651d2b9","Type":"ContainerDied","Data":"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"}
Sep 09 13:03:49.364744 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.364731    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx" event={"ID":"32bafd3b-7bde-4908-85ae-c6498651d2b9","Type":"ContainerDied","Data":"b39059e20ccde25d595e9a216e6db8985237e0d87d5f66d12fd7c43922167545"}
Sep 09 13:03:49.364880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.364772    2580 scope.go:117] "RemoveContainer" containerID="55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"
Sep 09 13:03:49.375186 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.375168    2580 scope.go:117] "RemoveContainer" containerID="55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"
Sep 09 13:03:49.375617 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:03:49.375553    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c\": container with ID starting with 55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c not found: ID does not exist" containerID="55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"
Sep 09 13:03:49.375617 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.375586    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c"} err="failed to get container status \"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c\": rpc error: code = NotFound desc = could not find container \"55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c\": container with ID starting with 55187a42492137b4be13edbf7219a5800b87208504cb2974c7b2dded0421992c not found: ID does not exist"
Sep 09 13:03:49.390134 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.390113    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:49.394213 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.394193    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-d0fb1d64-bf5d8b958-7kwsx"]
Sep 09 13:03:49.452486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.452398    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:49.458463 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.458435    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:49.458611 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.458557    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:49.548109 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.548076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:49.672839 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:49.672801    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:50.024012 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:50.023925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:50.227212 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:50.227171    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:50.453740 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:50.453706    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32bafd3b-7bde-4908-85ae-c6498651d2b9" path="/var/lib/kubelet/pods/32bafd3b-7bde-4908-85ae-c6498651d2b9/volumes"
Sep 09 13:03:50.458855 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:50.458824    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:50.458971 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:50.458877    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:51.459046 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:51.459006    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body=
Sep 09 13:03:51.459486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:51.459094    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused"
Sep 09 13:03:52.115600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.115565    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:52.308649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.308611    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:52.405472 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.405438    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:52.464341 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.464310    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:52.464831 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.464791    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:52.465846 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.465823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:03:52.559560 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.559524    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:52.660123 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.659999    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:52.760731 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:52.760700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:53.058789 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:53.058685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:53.154484 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:53.154450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:03:59.958584 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:03:59.958553    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:02.982130 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:02.982085    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:05.597249 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:05.597213    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:13.436182 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:13.436143    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:23.741658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:23.741616    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:23.813376 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:23.813336    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:23.891785 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:23.891745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:23.962756 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:23.962722    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:24.096156 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:24.096063    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:24.560301 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:24.560268    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:24.904916 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:24.904884    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:25.182834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:25.182755    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:26.934231 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:26.934191    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:27.032755 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:27.032726    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:28.178261 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:28.178219    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:28.276993 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:28.276961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:28.557959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:28.557879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:28.648257 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:28.648223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:28.979822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:28.979784    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:38.057265 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:38.057225    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:44.298747 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:44.298708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:49.108025 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:49.107939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:58.670457 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:58.670420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:58.770324 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:58.770284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:58.861263 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:58.861224    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:04:58.923753 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:04:58.923679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:00.092841 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:00.092795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:00.584974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:00.584935    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:00.889012 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:00.888914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:01.712437 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:01.712372    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:03.149122 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:03.149088    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:03.245812 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:03.245776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:04.307612 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:04.307574    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:04.405882 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:04.405843    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:04.513147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:04.513101    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:04.793102 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:04.793068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:04.875179 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:04.875144    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:10.479737 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:10.479701    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:20.788787 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.788751    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:05:20.789214 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.789013    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" podUID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" containerName="maas-api" containerID="cri-o://e62758cad47f5e7746a81924869fb66b33b165a14ed351fc8d7a4b4fb4dd83cf" gracePeriod=30
Sep 09 13:05:20.819261 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.819223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:20.838809 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.838782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:05:20.839037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.839010    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" podUID="b7030446-4937-4535-8c01-3a8396c4a342" containerName="payload-processing" containerID="cri-o://9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e" gracePeriod=30
Sep 09 13:05:20.919553 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.919520    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:20.954076 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.954042    2580 generic.go:356] "Generic (PLEG): container finished" podID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" containerID="e62758cad47f5e7746a81924869fb66b33b165a14ed351fc8d7a4b4fb4dd83cf" exitCode=0
Sep 09 13:05:20.954182 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:20.954120    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" event={"ID":"9082e4b4-8b7f-4d00-bbc2-48bb9885c088","Type":"ContainerDied","Data":"e62758cad47f5e7746a81924869fb66b33b165a14ed351fc8d7a4b4fb4dd83cf"}
Sep 09 13:05:21.013317 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.013169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:21.084658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.084606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:21.096255 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.096235    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:05:21.142844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.142818    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:05:21.220423 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6hgmf\" (UniqueName: \"kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf\") pod \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") "
Sep 09 13:05:21.220633 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220436    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") pod \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") "
Sep 09 13:05:21.220633 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs\") pod \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") "
Sep 09 13:05:21.220633 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume\") pod \"b7030446-4937-4535-8c01-3a8396c4a342\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") "
Sep 09 13:05:21.220633 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220595    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca\") pod \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\" (UID: \"9082e4b4-8b7f-4d00-bbc2-48bb9885c088\") "
Sep 09 13:05:21.220848 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220637    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vvx4\" (UniqueName: \"kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4\") pod \"b7030446-4937-4535-8c01-3a8396c4a342\" (UID: \"b7030446-4937-4535-8c01-3a8396c4a342\") "
Sep 09 13:05:21.221025 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.220987    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b7030446-4937-4535-8c01-3a8396c4a342" (UID: "b7030446-4937-4535-8c01-3a8396c4a342"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:21.221121 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.221079    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9082e4b4-8b7f-4d00-bbc2-48bb9885c088" (UID: "9082e4b4-8b7f-4d00-bbc2-48bb9885c088"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:05:21.222884 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.222854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9082e4b4-8b7f-4d00-bbc2-48bb9885c088" (UID: "9082e4b4-8b7f-4d00-bbc2-48bb9885c088"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:05:21.223164 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.223129    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9082e4b4-8b7f-4d00-bbc2-48bb9885c088" (UID: "9082e4b4-8b7f-4d00-bbc2-48bb9885c088"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:05:21.223241 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.223225    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4" (OuterVolumeSpecName: "kube-api-access-6vvx4") pod "b7030446-4937-4535-8c01-3a8396c4a342" (UID: "b7030446-4937-4535-8c01-3a8396c4a342"). InnerVolumeSpecName "kube-api-access-6vvx4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:21.223282 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.223250    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf" (OuterVolumeSpecName: "kube-api-access-6hgmf") pod "9082e4b4-8b7f-4d00-bbc2-48bb9885c088" (UID: "9082e4b4-8b7f-4d00-bbc2-48bb9885c088"). InnerVolumeSpecName "kube-api-access-6hgmf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321535    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6hgmf\" (UniqueName: \"kubernetes.io/projected/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-kube-api-access-6hgmf\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321557    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-maas-api-tls\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321568    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321576    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b7030446-4937-4535-8c01-3a8396c4a342-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321587    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9082e4b4-8b7f-4d00-bbc2-48bb9885c088-openshift-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.321594 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.321597    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vvx4\" (UniqueName: \"kubernetes.io/projected/b7030446-4937-4535-8c01-3a8396c4a342-kube-api-access-6vvx4\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:05:21.417671 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.417637    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:21.962539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.962487    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"
Sep 09 13:05:21.962958 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.962547    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt" event={"ID":"9082e4b4-8b7f-4d00-bbc2-48bb9885c088","Type":"ContainerDied","Data":"0f056e6d433314940e180314a914b5d312a3faa68e6c5b058d936155833739ea"}
Sep 09 13:05:21.962958 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.962611    2580 scope.go:117] "RemoveContainer" containerID="e62758cad47f5e7746a81924869fb66b33b165a14ed351fc8d7a4b4fb4dd83cf"
Sep 09 13:05:21.964116 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.964096    2580 generic.go:356] "Generic (PLEG): container finished" podID="b7030446-4937-4535-8c01-3a8396c4a342" containerID="9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e" exitCode=0
Sep 09 13:05:21.964197 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.964144    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"
Sep 09 13:05:21.964197 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.964178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" event={"ID":"b7030446-4937-4535-8c01-3a8396c4a342","Type":"ContainerDied","Data":"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"}
Sep 09 13:05:21.964311 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.964230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r" event={"ID":"b7030446-4937-4535-8c01-3a8396c4a342","Type":"ContainerDied","Data":"dff449c30916d875ef1487f926c3449faf5d713a2d9fc1d2da4025b93a20a137"}
Sep 09 13:05:21.972968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.972949    2580 scope.go:117] "RemoveContainer" containerID="9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"
Sep 09 13:05:21.983445 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.983426    2580 scope.go:117] "RemoveContainer" containerID="9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"
Sep 09 13:05:21.983717 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:05:21.983698    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e\": container with ID starting with 9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e not found: ID does not exist" containerID="9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"
Sep 09 13:05:21.983780 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.983725    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e"} err="failed to get container status \"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e\": rpc error: code = NotFound desc = could not find container \"9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e\": container with ID starting with 9053bb49d0480bf6ee22018af0e8d6a9a792a5aff29f3b0c3a9a7cf2b83db18e not found: ID does not exist"
Sep 09 13:05:21.989346 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.989324    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:05:21.992972 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:21.992954    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-autores-b4ea80-7969b8f77-7td4r"]
Sep 09 13:05:22.006801 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:22.006780    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:05:22.011210 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:22.011187    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b4ea80-64f88c444-6l4xt"]
Sep 09 13:05:22.454056 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:22.454027    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" path="/var/lib/kubelet/pods/9082e4b4-8b7f-4d00-bbc2-48bb9885c088/volumes"
Sep 09 13:05:22.454401 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:22.454388    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b7030446-4937-4535-8c01-3a8396c4a342" path="/var/lib/kubelet/pods/b7030446-4937-4535-8c01-3a8396c4a342/volumes"
Sep 09 13:05:24.398677 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:24.398638    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:25.300158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:25.300120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:26.426999 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:26.426947    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:34.781018 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:34.780986    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:34.880897 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:34.880861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:34.964604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:34.964575    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:35.020730 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:35.020698    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:35.127469 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:35.127385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:35.883921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:35.883877    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:35.984912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:35.984882    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:36.082327 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:36.082292    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:37.595414 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:37.595364    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:43.318085 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:43.318048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:43.374462 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:43.374432    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:43.446357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:43.446318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:43.545780 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:43.545745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:43.646032 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:43.646004    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:44.074081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:44.074010    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:05:57.286761 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:05:57.286719    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:00.159125 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:00.159081    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:03.327810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:03.327741    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:03.590850 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:03.590764    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:06:03.591023 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:03.590994    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" containerID="cri-o://e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f" gracePeriod=30
Sep 09 13:06:03.609082 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:03.609040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:04.067366 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:04.067324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:04.412976 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:04.412944    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:05.359034 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:05.358981    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:07.089000 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:07.088970    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:07.190825 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:07.190792    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:07.467476 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:07.467438    2580 patch_prober.go:28] interesting pod/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 13:06:07.467672 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:07.467545    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 13:06:08.229277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.229247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:08.329237 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.329198    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:08.428608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.428573    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:08.711978 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.711906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:08.789828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.789794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:08.854051 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.853992    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:06:08.933245 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933217    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933255    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933295    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933330    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933355    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933389    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933438    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933465    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wjlkz\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket\") pod \"062a7568-3680-4eae-9718-678cf1ae344b\" (UID: \"062a7568-3680-4eae-9718-678cf1ae344b\") "
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933633    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data" (OuterVolumeSpecName: "istio-data") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933646    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:06:08.933699 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933659    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:06:08.934028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:06:08.934028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933850    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:08.934028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933871    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-data\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:08.934028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933884    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-credential-socket\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:08.934028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.933843    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:06:08.935485 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.935461    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:06:08.935617 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.935580    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token" (OuterVolumeSpecName: "istio-token") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:06:08.936004 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.935984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:06:08.936042 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:08.936029    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz" (OuterVolumeSpecName: "kube-api-access-wjlkz") pod "062a7568-3680-4eae-9718-678cf1ae344b" (UID: "062a7568-3680-4eae-9718-678cf1ae344b"). InnerVolumeSpecName "kube-api-access-wjlkz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:06:09.034691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034656    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-istio-token\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.034691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034684    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/062a7568-3680-4eae-9718-678cf1ae344b-istiod-ca-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.034691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034695    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-istio-envoy\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.034919 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034704    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wjlkz\" (UniqueName: \"kubernetes.io/projected/062a7568-3680-4eae-9718-678cf1ae344b-kube-api-access-wjlkz\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.034919 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034713    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/062a7568-3680-4eae-9718-678cf1ae344b-workload-socket\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.034919 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.034721    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/062a7568-3680-4eae-9718-678cf1ae344b-istio-podinfo\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:06:09.249010 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.248975    2580 generic.go:356] "Generic (PLEG): container finished" podID="062a7568-3680-4eae-9718-678cf1ae344b" containerID="e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f" exitCode=0
Sep 09 13:06:09.249520 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.249032    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"
Sep 09 13:06:09.249520 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.249061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerDied","Data":"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"}
Sep 09 13:06:09.249520 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.249121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl" event={"ID":"062a7568-3680-4eae-9718-678cf1ae344b","Type":"ContainerDied","Data":"42b9b7aefee938ddb4505a7656583e957ffed76c85149f56fa9e943ebfab3d36"}
Sep 09 13:06:09.249520 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.249144    2580 scope.go:117] "RemoveContainer" containerID="e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"
Sep 09 13:06:09.259308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.259292    2580 scope.go:117] "RemoveContainer" containerID="5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"
Sep 09 13:06:09.267603 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.267569    2580 scope.go:117] "RemoveContainer" containerID="e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"
Sep 09 13:06:09.267904 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:06:09.267884    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f\": container with ID starting with e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f not found: ID does not exist" containerID="e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"
Sep 09 13:06:09.268009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.267909    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f"} err="failed to get container status \"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f\": rpc error: code = NotFound desc = could not find container \"e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f\": container with ID starting with e09ce57d85591942eaaa3c913a3ac91f9fcf9cc36e24e5b4a2b577b5e98fa74f not found: ID does not exist"
Sep 09 13:06:09.268009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.267928    2580 scope.go:117] "RemoveContainer" containerID="5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"
Sep 09 13:06:09.268251 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:06:09.268231    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865\": container with ID starting with 5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865 not found: ID does not exist" containerID="5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"
Sep 09 13:06:09.268328 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.268260    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865"} err="failed to get container status \"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865\": rpc error: code = NotFound desc = could not find container \"5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865\": container with ID starting with 5336202fd0130029e1d2ee047cab5f0beec398ad84c70d4a7d371abf9dc76865 not found: ID does not exist"
Sep 09 13:06:09.271771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.271749    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:06:09.275433 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:09.275413    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-b4ea80-openshift-default-5c85b67954-nbqhl"]
Sep 09 13:06:10.097379 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:10.097345    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:10.452984 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:10.452950    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="062a7568-3680-4eae-9718-678cf1ae344b" path="/var/lib/kubelet/pods/062a7568-3680-4eae-9718-678cf1ae344b/volumes"
Sep 09 13:06:15.741731 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:15.741640    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:15.849245 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:15.849208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:15.942272 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:15.942242    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:18.701595 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:18.701561    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:22.397083 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:22.397051    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:23.958037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958002    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:06:23.958661 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958644    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b7030446-4937-4535-8c01-3a8396c4a342" containerName="payload-processing"
Sep 09 13:06:23.958722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958665    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b7030446-4937-4535-8c01-3a8396c4a342" containerName="payload-processing"
Sep 09 13:06:23.958722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958700    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32bafd3b-7bde-4908-85ae-c6498651d2b9" containerName="payload-pre-processing"
Sep 09 13:06:23.958722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958710    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="32bafd3b-7bde-4908-85ae-c6498651d2b9" containerName="payload-pre-processing"
Sep 09 13:06:23.958722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958721    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:06:23.958858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958732    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:06:23.958858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958750    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:06:23.958858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958759    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:06:23.958858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958768    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" containerName="maas-api"
Sep 09 13:06:23.958858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958776    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" containerName="maas-api"
Sep 09 13:06:23.959024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958871    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:06:23.959024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958882    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="32bafd3b-7bde-4908-85ae-c6498651d2b9" containerName="payload-pre-processing"
Sep 09 13:06:23.959024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958892    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b7030446-4937-4535-8c01-3a8396c4a342" containerName="payload-processing"
Sep 09 13:06:23.959024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.958905    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9082e4b4-8b7f-4d00-bbc2-48bb9885c088" containerName="maas-api"
Sep 09 13:06:23.963625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.963601    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:23.966038 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.965928    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c991b\"/\"kube-root-ca.crt\""
Sep 09 13:06:23.966945 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.966924    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c991b\"/\"default-dockercfg-22dgq\""
Sep 09 13:06:23.967125 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.967103    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:06:23.967286 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.967221    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-7c991b\"/\"openshift-service-ca.crt\""
Sep 09 13:06:23.967286 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.967226    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-7c991b\"/\"test-model-7c991b-kserve-self-signed-certs\""
Sep 09 13:06:23.973976 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:23.973950    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:24.065159 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.065159 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.065354 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vm2p\" (UniqueName: \"kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.065354 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065265    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.065354 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.065456 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.065353    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166011 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.165974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vm2p\" (UniqueName: \"kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166045    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166113    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166366 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166179    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166393    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166434    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.166575 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.166536    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.168332 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.168311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.168600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.168581    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.173363 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.173341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vm2p\" (UniqueName: \"kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p\") pod \"test-model-7c991b-kserve-74cd854555-8fst2\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") " pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.275834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.275765    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:24.547075 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:24.547047    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:06:24.548081 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:06:24.548054    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca3d3cc4_62e3_49f3_bf4f_c5d41fff3546.slice/crio-3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6 WatchSource:0}: Error finding container 3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6: Status 404 returned error can't find the container with id 3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6
Sep 09 13:06:25.358236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:25.358195    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" event={"ID":"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546","Type":"ContainerStarted","Data":"397421f3cb0616f1623fd4cb6d611652e3ec94b804d249883ef097d9d922fbc9"}
Sep 09 13:06:25.358236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:25.358240    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" event={"ID":"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546","Type":"ContainerStarted","Data":"3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6"}
Sep 09 13:06:25.374962 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:25.374916    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" podStartSLOduration=2.374899129 podStartE2EDuration="2.374899129s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:06:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:06:25.374067459 +0000 UTC m=+1083.409315366" watchObservedRunningTime="2026-09-09 13:06:25.374899129 +0000 UTC m=+1083.410147039"
Sep 09 13:06:25.791400 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:25.791367    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:26.033126 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:26.033078    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:26.085755 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:26.085663    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:31.906665 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:31.906624    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:32.197103 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:32.197014    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:34.276300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:34.276269    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:34.276717 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:34.276327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:34.289450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:34.289426    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:34.425342 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:34.425311    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:06:37.828530 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:37.828466    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:37.928705 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:37.928668    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:38.028897 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:38.028866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:38.130306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:38.130226    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:40.967882 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:40.967852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:44.166914 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:44.166878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:48.789956 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:48.789906    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:49.728736 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:49.728700    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:55.014557 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:55.014489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:55.116723 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:55.116688    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:55.216317 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:55.216284    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:55.289348 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:55.289272    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:55.670318 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:55.670283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:56.061462 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:56.061384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:56.168060 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:56.168019    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:56.261348 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:56.261316    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:56.362190 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:56.362113    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:57.100458 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:57.100417    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:57.251737 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:57.251708    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:57.348763 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:57.348729    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:06:59.102020 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:06:59.101973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:01.611058 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:01.611023    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:01.711114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:01.711075    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:01.950275 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:01.950234    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:05.410567 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:05.410529    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:05.510485 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:05.510440    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:06.037416 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:06.037380    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:11.778302 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:11.778255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:11.878898 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:11.878861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:12.012669 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:12.012634    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:12.347431 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:12.347400    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:12.476717 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:12.476681    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:18.487357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:18.487324    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:18.542034 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:18.542006    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:22.268012 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:22.267973    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:26.175352 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:26.175318    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:26.301578 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:26.301530    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:26.546641 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:26.546495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:26.668631 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:26.668593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:29.574602 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:29.574559    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:30.556664 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:30.556629    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:07:30.568348 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:30.568313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:30.809752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:30.809660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:31.739682 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:31.739644    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:38.437625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:38.437578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:40.473297 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:40.473267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:40.971284 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:40.971255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:44.144075 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:44.144036    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:44.200497 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:44.200460    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:45.576881 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.576760    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" podUID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" containerName="main" containerID="cri-o://397421f3cb0616f1623fd4cb6d611652e3ec94b804d249883ef097d9d922fbc9" gracePeriod=45
Sep 09 13:07:45.828001 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.827924    2580 generic.go:356] "Generic (PLEG): container finished" podID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" containerID="397421f3cb0616f1623fd4cb6d611652e3ec94b804d249883ef097d9d922fbc9" exitCode=0
Sep 09 13:07:45.828133 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.828006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" event={"ID":"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546","Type":"ContainerDied","Data":"397421f3cb0616f1623fd4cb6d611652e3ec94b804d249883ef097d9d922fbc9"}
Sep 09 13:07:45.828133 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.828047    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2" event={"ID":"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546","Type":"ContainerDied","Data":"3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6"}
Sep 09 13:07:45.828133 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.828058    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3aae368dab0254854f5171d1bec4cb69529d5ce62d97164e70c0d6af592c8ce6"
Sep 09 13:07:45.828512 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.828484    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:07:45.875440 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875409    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875471    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875495    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vm2p\" (UniqueName: \"kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875538    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875594    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875836 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875625    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs\") pod \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\" (UID: \"ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546\") "
Sep 09 13:07:45.875836 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875733    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home" (OuterVolumeSpecName: "home") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:45.875943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875894    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache" (OuterVolumeSpecName: "model-cache") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:45.875943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875911    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:45.876058 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.875994    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.876058 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.876013    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.876058 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.876028    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.877668 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.877642    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm" (OuterVolumeSpecName: "dshm") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:07:45.877783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.877706    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p" (OuterVolumeSpecName: "kube-api-access-4vm2p") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "kube-api-access-4vm2p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:07:45.877783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.877710    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" (UID: "ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:07:45.977074 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.977051    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vm2p\" (UniqueName: \"kubernetes.io/projected/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-kube-api-access-4vm2p\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.977074 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.977072    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:45.977216 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:45.977081    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:07:46.833213 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:46.833135    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"
Sep 09 13:07:46.848853 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:46.848829    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:07:46.852643 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:46.852621    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-7c991b/test-model-7c991b-kserve-74cd854555-8fst2"]
Sep 09 13:07:47.386548 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:47.386489    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:47.452780 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:47.452749    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:47.913343 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:47.913308    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:48.454263 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:48.454229    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" path="/var/lib/kubelet/pods/ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546/volumes"
Sep 09 13:07:48.801684 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:48.801606    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:48.859874 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:48.859831    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:49.067580 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:49.067465    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:49.512975 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:49.512939    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:53.468441 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:53.468376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:57.921637 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:57.921597    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:58.443609 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:58.443578    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:58.542599 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:58.542560    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:58.636655 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:58.636618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:58.697675 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:58.697603    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:59.016452 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:59.016360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:07:59.560878 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:07:59.560844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:00.145972 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.145939    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:08:00.146522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.146190    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" podUID="963e57b9-4908-4be3-aa79-7c1225647ff3" containerName="maas-api" containerID="cri-o://33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23" gracePeriod=30
Sep 09 13:08:00.171651 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.171618    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:00.271997 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.271963    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:00.364531 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.364479    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:00.423847 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.423490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:00.595491 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.595463    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:08:00.696786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.696710    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vb4t9\" (UniqueName: \"kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9\") pod \"963e57b9-4908-4be3-aa79-7c1225647ff3\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") "
Sep 09 13:08:00.696786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.696741    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca\") pod \"963e57b9-4908-4be3-aa79-7c1225647ff3\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") "
Sep 09 13:08:00.696786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.696759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs\") pod \"963e57b9-4908-4be3-aa79-7c1225647ff3\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") "
Sep 09 13:08:00.697016 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.696891    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls\") pod \"963e57b9-4908-4be3-aa79-7c1225647ff3\" (UID: \"963e57b9-4908-4be3-aa79-7c1225647ff3\") "
Sep 09 13:08:00.697142 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.697113    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "963e57b9-4908-4be3-aa79-7c1225647ff3" (UID: "963e57b9-4908-4be3-aa79-7c1225647ff3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:08:00.698934 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.698901    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "963e57b9-4908-4be3-aa79-7c1225647ff3" (UID: "963e57b9-4908-4be3-aa79-7c1225647ff3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:08:00.698934 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.698928    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9" (OuterVolumeSpecName: "kube-api-access-vb4t9") pod "963e57b9-4908-4be3-aa79-7c1225647ff3" (UID: "963e57b9-4908-4be3-aa79-7c1225647ff3"). InnerVolumeSpecName "kube-api-access-vb4t9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:08:00.699084 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.698989    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "963e57b9-4908-4be3-aa79-7c1225647ff3" (UID: "963e57b9-4908-4be3-aa79-7c1225647ff3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:08:00.798432 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.798396    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vb4t9\" (UniqueName: \"kubernetes.io/projected/963e57b9-4908-4be3-aa79-7c1225647ff3-kube-api-access-vb4t9\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.798432 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.798430    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/963e57b9-4908-4be3-aa79-7c1225647ff3-openshift-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.798432 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.798440    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.798653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.798449    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/963e57b9-4908-4be3-aa79-7c1225647ff3-maas-api-tls\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:08:00.915076 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.915038    2580 generic.go:356] "Generic (PLEG): container finished" podID="963e57b9-4908-4be3-aa79-7c1225647ff3" containerID="33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23" exitCode=0
Sep 09 13:08:00.915254 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.915080    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"
Sep 09 13:08:00.915254 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.915121    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" event={"ID":"963e57b9-4908-4be3-aa79-7c1225647ff3","Type":"ContainerDied","Data":"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"}
Sep 09 13:08:00.915254 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.915173    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp" event={"ID":"963e57b9-4908-4be3-aa79-7c1225647ff3","Type":"ContainerDied","Data":"a5f1ff6d7b3a5d687a96d0a9f2ee533986850861594fc11779170be11bed4701"}
Sep 09 13:08:00.915254 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.915213    2580 scope.go:117] "RemoveContainer" containerID="33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"
Sep 09 13:08:00.925315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.925299    2580 scope.go:117] "RemoveContainer" containerID="33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"
Sep 09 13:08:00.925592 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:08:00.925574    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23\": container with ID starting with 33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23 not found: ID does not exist" containerID="33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"
Sep 09 13:08:00.925672 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.925605    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23"} err="failed to get container status \"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23\": rpc error: code = NotFound desc = could not find container \"33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23\": container with ID starting with 33be5daac35fbee2821b209fbb73d437bf4802f9efc9e5773ee2cf89b1ee9a23 not found: ID does not exist"
Sep 09 13:08:00.936004 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.935983    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:08:00.939623 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:00.939599    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-7c991b-d66495686-5k4pp"]
Sep 09 13:08:02.452947 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:02.452902    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="963e57b9-4908-4be3-aa79-7c1225647ff3" path="/var/lib/kubelet/pods/963e57b9-4908-4be3-aa79-7c1225647ff3/volumes"
Sep 09 13:08:04.154935 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:04.154895    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:04.654845 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:04.654806    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:04.756388 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:04.756354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:04.951048 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:04.950961    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:05.332110 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:05.332032    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:05.423842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:05.423800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:05.551404 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:05.551357    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:07.154096 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:07.154061    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:07.212014 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:07.211965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:07.413844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:07.413747    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:07.993686 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:07.993647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:08.314939 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:08.314829    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:08.500786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:08.500745    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:09.019070 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:09.019040    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:11.494940 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:11.494910    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:14.796546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:14.796490    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:14.853330 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:14.853299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:17.654683 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:17.654647    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:17.716752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:17.716716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:18.633268 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:18.633235    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:18.946458 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:18.946424    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:19.131579 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:19.131546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:21.839664 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:21.839622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:21.937816 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:21.937783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:22.513893 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:22.513867    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:08:22.514257 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:22.514232    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:08:23.107411 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:23.107376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:23.208257 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:23.208222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:23.564930 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:23.564891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:23.656821 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:23.656783    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:33.874607 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:33.874564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:36.856584 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:36.856541    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:37.223483 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:37.223418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:37.567761 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:37.567671    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:37.724768 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:37.724735    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:40.987419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:40.987385    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:41.086831 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:41.086794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:42.139909 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:42.139860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:42.239257 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:42.239223    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:42.343079 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:42.343043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:42.619546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:42.619482    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:42.717267 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:42.717222    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:44.887045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:44.886913    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:46.923946 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:46.923912    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:50.753338 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:50.753279    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:52.221752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.221710    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:52.282361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.282328    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:08:52.282887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.282839    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="963e57b9-4908-4be3-aa79-7c1225647ff3" containerName="maas-api"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.282902    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="963e57b9-4908-4be3-aa79-7c1225647ff3" containerName="maas-api"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.282951    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" containerName="main"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.282962    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" containerName="main"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.283078    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca3d3cc4-62e3-49f3-bf4f-c5d41fff3546" containerName="main"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.283099    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="062a7568-3680-4eae-9718-678cf1ae344b" containerName="istio-proxy"
Sep 09 13:08:52.283135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.283110    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="963e57b9-4908-4be3-aa79-7c1225647ff3" containerName="maas-api"
Sep 09 13:08:52.286249 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.286233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.288481 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.288459    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-941953\""
Sep 09 13:08:52.300039 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.300016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:08:52.301844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.301821    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:08:52.349736 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.349702    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gbm\" (UniqueName: \"kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.349876 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.349856    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.450342 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.450309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.450526 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.450355    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gbm\" (UniqueName: \"kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.451061 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.451035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.457464 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.457437    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gbm\" (UniqueName: \"kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm\") pod \"payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.597184 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.597110    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:52.871281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:52.871249    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:08:52.872135 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:08:52.872108    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc529b246_dd89_4741_a7df_123e4c3c1098.slice/crio-034238dee90a391ef67419b473b2c34b30510eef6dde063ff16d4aee4ca7cc83 WatchSource:0}: Error finding container 034238dee90a391ef67419b473b2c34b30510eef6dde063ff16d4aee4ca7cc83: Status 404 returned error can't find the container with id 034238dee90a391ef67419b473b2c34b30510eef6dde063ff16d4aee4ca7cc83
Sep 09 13:08:53.219052 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:53.219022    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" event={"ID":"c529b246-dd89-4741-a7df-123e4c3c1098","Type":"ContainerStarted","Data":"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"}
Sep 09 13:08:53.219270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:53.219078    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" event={"ID":"c529b246-dd89-4741-a7df-123e4c3c1098","Type":"ContainerStarted","Data":"034238dee90a391ef67419b473b2c34b30510eef6dde063ff16d4aee4ca7cc83"}
Sep 09 13:08:53.219270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:53.219144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:08:53.233673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:08:53.233631    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" podStartSLOduration=1.233616463 podStartE2EDuration="1.233616463s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:08:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:08:53.232764127 +0000 UTC m=+1231.268012037" watchObservedRunningTime="2026-09-09 13:08:53.233616463 +0000 UTC m=+1231.268864369"
Sep 09 13:09:04.003305 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:04.003257    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:04.226043 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:04.226014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:09:06.029105 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:06.029076    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:10.659614 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.659574    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:09:10.663693 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.663659    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.667316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.667151    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"kube-root-ca.crt\""
Sep 09 13:09:10.667316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.667165    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"auth-test-model-ada1ae-kserve-self-signed-certs\""
Sep 09 13:09:10.667316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.667183    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"default-dockercfg-7xt76\""
Sep 09 13:09:10.667316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.667186    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"openshift-service-ca.crt\""
Sep 09 13:09:10.671733 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.671712    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:09:10.687187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.687156    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:10.807720 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807693    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.807880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807728    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4wlr\" (UniqueName: \"kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.807880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.807880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807842    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.807880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807871    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.808049 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.807912    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908458 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908423    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908458 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908460    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908713 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908475    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908713 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908602    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908713 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908685    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908723    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4wlr\" (UniqueName: \"kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.908887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.909059 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.908928    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.911061 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.910971    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.911453 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.911435    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.915861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.915840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4wlr\" (UniqueName: \"kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr\") pod \"auth-test-model-ada1ae-kserve-9465b64f5-dq6z9\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:10.977200 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:10.977158    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:11.251042 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:11.251015    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:09:11.251986 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:11.251964    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2612ef6e_2762_4236_b5e0_b8e1c4b3d894.slice/crio-da22fb647123cd8de9d178f57bfb259b5023b8af8d82813e0c666978b53d107d WatchSource:0}: Error finding container da22fb647123cd8de9d178f57bfb259b5023b8af8d82813e0c666978b53d107d: Status 404 returned error can't find the container with id da22fb647123cd8de9d178f57bfb259b5023b8af8d82813e0c666978b53d107d
Sep 09 13:09:11.327622 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:11.327583    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" event={"ID":"2612ef6e-2762-4236-b5e0-b8e1c4b3d894","Type":"ContainerStarted","Data":"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"}
Sep 09 13:09:11.327773 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:11.327630    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" event={"ID":"2612ef6e-2762-4236-b5e0-b8e1c4b3d894","Type":"ContainerStarted","Data":"da22fb647123cd8de9d178f57bfb259b5023b8af8d82813e0c666978b53d107d"}
Sep 09 13:09:11.343709 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:11.343660    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" podStartSLOduration=1.3436446260000001 podStartE2EDuration="1.343644626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:09:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:09:11.342411726 +0000 UTC m=+1249.377659678" watchObservedRunningTime="2026-09-09 13:09:11.343644626 +0000 UTC m=+1249.378892533"
Sep 09 13:09:12.561754 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:12.561720    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:15.488198 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:15.488111    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:16.174103 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.174068    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:16.177434 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.177417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:16.179807 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.179785    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s2rqh\""
Sep 09 13:09:16.181796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.181771    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:16.254184 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.254155    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tjhf\" (UniqueName: \"kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf\") pod \"test-curl-7843-139839173695216\" (UID: \"11c309a6-2412-435a-9d89-587702db46d9\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:16.355384 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.355359    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tjhf\" (UniqueName: \"kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf\") pod \"test-curl-7843-139839173695216\" (UID: \"11c309a6-2412-435a-9d89-587702db46d9\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:16.362863 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.362835    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tjhf\" (UniqueName: \"kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf\") pod \"test-curl-7843-139839173695216\" (UID: \"11c309a6-2412-435a-9d89-587702db46d9\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:16.489775 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.489698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:16.760496 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.760471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:16.762291 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:16.762262    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod11c309a6_2412_435a_9d89_587702db46d9.slice/crio-8f9ce2c07bede06a3b02f3135ae8f18d5baa8e39bfb4a9b054679bf1ebc4df96 WatchSource:0}: Error finding container 8f9ce2c07bede06a3b02f3135ae8f18d5baa8e39bfb4a9b054679bf1ebc4df96: Status 404 returned error can't find the container with id 8f9ce2c07bede06a3b02f3135ae8f18d5baa8e39bfb4a9b054679bf1ebc4df96
Sep 09 13:09:16.763539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:16.763495    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:09:17.368483 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:17.368449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695216" event={"ID":"11c309a6-2412-435a-9d89-587702db46d9","Type":"ContainerStarted","Data":"8f9ce2c07bede06a3b02f3135ae8f18d5baa8e39bfb4a9b054679bf1ebc4df96"}
Sep 09 13:09:18.375380 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:18.375346    2580 generic.go:356] "Generic (PLEG): container finished" podID="11c309a6-2412-435a-9d89-587702db46d9" containerID="0ed4a1840a10f8dcec76e192cc8cd726b7a96bccbaea5dcb6fc54723e478098a" exitCode=0
Sep 09 13:09:18.375809 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:18.375434    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695216" event={"ID":"11c309a6-2412-435a-9d89-587702db46d9","Type":"ContainerDied","Data":"0ed4a1840a10f8dcec76e192cc8cd726b7a96bccbaea5dcb6fc54723e478098a"}
Sep 09 13:09:19.510834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.510812    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:19.528538 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.528479    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-139839173695216_11c309a6-2412-435a-9d89-587702db46d9/test-curl-7843-139839173695216/0.log"
Sep 09 13:09:19.549123 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.549096    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:19.553076 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.553053    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:19.582563 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.582541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tjhf\" (UniqueName: \"kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf\") pod \"11c309a6-2412-435a-9d89-587702db46d9\" (UID: \"11c309a6-2412-435a-9d89-587702db46d9\") "
Sep 09 13:09:19.584566 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.584543    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf" (OuterVolumeSpecName: "kube-api-access-4tjhf") pod "11c309a6-2412-435a-9d89-587702db46d9" (UID: "11c309a6-2412-435a-9d89-587702db46d9"). InnerVolumeSpecName "kube-api-access-4tjhf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:09:19.683769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.683739    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tjhf\" (UniqueName: \"kubernetes.io/projected/11c309a6-2412-435a-9d89-587702db46d9-kube-api-access-4tjhf\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:09:19.685159 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.685135    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:19.685630 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.685613    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11c309a6-2412-435a-9d89-587702db46d9" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:19.685716 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.685632    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="11c309a6-2412-435a-9d89-587702db46d9" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:19.685779 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.685742    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="11c309a6-2412-435a-9d89-587702db46d9" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:19.688952 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.688920    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:19.692121 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.692097    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:19.784826 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.784800    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-262r5\" (UniqueName: \"kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5\") pod \"test-curl-7843-139839173695216\" (UID: \"e0570256-f560-4d55-a87d-6e59af38f31e\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:19.886112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.886081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-262r5\" (UniqueName: \"kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5\") pod \"test-curl-7843-139839173695216\" (UID: \"e0570256-f560-4d55-a87d-6e59af38f31e\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:19.893746 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:19.893719    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-262r5\" (UniqueName: \"kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5\") pod \"test-curl-7843-139839173695216\" (UID: \"e0570256-f560-4d55-a87d-6e59af38f31e\") " pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:20.000153 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.000080    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:20.268698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.268671    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:20.269984 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:20.269953    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode0570256_f560_4d55_a87d_6e59af38f31e.slice/crio-e79b81d9d19e8ccfdfa256cb535169fd6fc8a74929a135c0a76ff732384aabfa WatchSource:0}: Error finding container e79b81d9d19e8ccfdfa256cb535169fd6fc8a74929a135c0a76ff732384aabfa: Status 404 returned error can't find the container with id e79b81d9d19e8ccfdfa256cb535169fd6fc8a74929a135c0a76ff732384aabfa
Sep 09 13:09:20.394810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.394771    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695216" event={"ID":"e0570256-f560-4d55-a87d-6e59af38f31e","Type":"ContainerStarted","Data":"e79b81d9d19e8ccfdfa256cb535169fd6fc8a74929a135c0a76ff732384aabfa"}
Sep 09 13:09:20.396096 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.396075    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:20.396203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.396099    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8f9ce2c07bede06a3b02f3135ae8f18d5baa8e39bfb4a9b054679bf1ebc4df96"
Sep 09 13:09:20.407272 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.407243    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7843-139839173695216" oldPodUID="11c309a6-2412-435a-9d89-587702db46d9" podUID="e0570256-f560-4d55-a87d-6e59af38f31e"
Sep 09 13:09:20.452837 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.452813    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11c309a6-2412-435a-9d89-587702db46d9" path="/var/lib/kubelet/pods/11c309a6-2412-435a-9d89-587702db46d9/volumes"
Sep 09 13:09:20.977814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.977781    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:20.978272 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.977831    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:20.990493 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:20.990465    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:21.415191 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:21.415161    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:09:22.622692 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:22.622648    2580 scope.go:117] "RemoveContainer" containerID="dc74b85e853823093cc9e248637c7207d56c51bd75c3313629d9b44ef8f04f07"
Sep 09 13:09:24.422552 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:24.422486    2580 generic.go:356] "Generic (PLEG): container finished" podID="e0570256-f560-4d55-a87d-6e59af38f31e" containerID="14ed548262dab8cc455a056a4f0f9c2adc1fd0b44eedf5859c90caf3083da0c2" exitCode=0
Sep 09 13:09:24.422932 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:24.422579    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695216" event={"ID":"e0570256-f560-4d55-a87d-6e59af38f31e","Type":"ContainerDied","Data":"14ed548262dab8cc455a056a4f0f9c2adc1fd0b44eedf5859c90caf3083da0c2"}
Sep 09 13:09:25.564616 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.564594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:25.585463 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.585401    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-139839173695216_e0570256-f560-4d55-a87d-6e59af38f31e/test-curl-7843-139839173695216/0.log"
Sep 09 13:09:25.607112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.607087    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:25.610722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.610695    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-139839173695216"]
Sep 09 13:09:25.635841 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.635819    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-262r5\" (UniqueName: \"kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5\") pod \"e0570256-f560-4d55-a87d-6e59af38f31e\" (UID: \"e0570256-f560-4d55-a87d-6e59af38f31e\") "
Sep 09 13:09:25.637767 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.637740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5" (OuterVolumeSpecName: "kube-api-access-262r5") pod "e0570256-f560-4d55-a87d-6e59af38f31e" (UID: "e0570256-f560-4d55-a87d-6e59af38f31e"). InnerVolumeSpecName "kube-api-access-262r5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:09:25.737203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.737135    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-262r5\" (UniqueName: \"kubernetes.io/projected/e0570256-f560-4d55-a87d-6e59af38f31e-kube-api-access-262r5\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:09:25.737471 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.737450    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:25.737868 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.737856    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0570256-f560-4d55-a87d-6e59af38f31e" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:25.737912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.737878    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0570256-f560-4d55-a87d-6e59af38f31e" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:25.737974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.737965    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0570256-f560-4d55-a87d-6e59af38f31e" containerName="test-curl-7843-139839173695216"
Sep 09 13:09:25.741160 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.741144    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:25.747886 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.747854    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:25.838083 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.838052    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wll5d\" (UniqueName: \"kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d\") pod \"test-curl-7843-139839173695792\" (UID: \"b73546bb-ca8e-42bb-81a5-615e93bf16b0\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:25.938553 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.938523    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wll5d\" (UniqueName: \"kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d\") pod \"test-curl-7843-139839173695792\" (UID: \"b73546bb-ca8e-42bb-81a5-615e93bf16b0\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:25.945966 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:25.945939    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wll5d\" (UniqueName: \"kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d\") pod \"test-curl-7843-139839173695792\" (UID: \"b73546bb-ca8e-42bb-81a5-615e93bf16b0\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:26.052581 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.052526    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:26.320193 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.320156    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:26.321566 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:26.321542    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb73546bb_ca8e_42bb_81a5_615e93bf16b0.slice/crio-1c5b366d3936184a978359930d23d11184cfe93a6ae753fa6cfeb80f9d06086c WatchSource:0}: Error finding container 1c5b366d3936184a978359930d23d11184cfe93a6ae753fa6cfeb80f9d06086c: Status 404 returned error can't find the container with id 1c5b366d3936184a978359930d23d11184cfe93a6ae753fa6cfeb80f9d06086c
Sep 09 13:09:26.435487 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.435457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695792" event={"ID":"b73546bb-ca8e-42bb-81a5-615e93bf16b0","Type":"ContainerStarted","Data":"1c5b366d3936184a978359930d23d11184cfe93a6ae753fa6cfeb80f9d06086c"}
Sep 09 13:09:26.436822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.436800    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e79b81d9d19e8ccfdfa256cb535169fd6fc8a74929a135c0a76ff732384aabfa"
Sep 09 13:09:26.436822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.436804    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:26.446961 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:09:26.446934    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7843-139839173695216\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="e0570256-f560-4d55-a87d-6e59af38f31e" pod="opendatahub/test-curl-7843-139839173695216"
Sep 09 13:09:26.453161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:26.453140    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0570256-f560-4d55-a87d-6e59af38f31e" path="/var/lib/kubelet/pods/e0570256-f560-4d55-a87d-6e59af38f31e/volumes"
Sep 09 13:09:27.444377 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:27.444335    2580 generic.go:356] "Generic (PLEG): container finished" podID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" containerID="dcae99ee0b109be5f425eab9621611f1c84e4e634ef8fd77855110045a6f904a" exitCode=0
Sep 09 13:09:27.444816 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:27.444389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695792" event={"ID":"b73546bb-ca8e-42bb-81a5-615e93bf16b0","Type":"ContainerDied","Data":"dcae99ee0b109be5f425eab9621611f1c84e4e634ef8fd77855110045a6f904a"}
Sep 09 13:09:28.581715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.581690    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:28.600496 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.600468    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-139839173695792_b73546bb-ca8e-42bb-81a5-615e93bf16b0/test-curl-7843-139839173695792/0.log"
Sep 09 13:09:28.622038 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.622015    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:28.624357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.624327    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:28.662147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.662125    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wll5d\" (UniqueName: \"kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d\") pod \"b73546bb-ca8e-42bb-81a5-615e93bf16b0\" (UID: \"b73546bb-ca8e-42bb-81a5-615e93bf16b0\") "
Sep 09 13:09:28.664140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.664114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d" (OuterVolumeSpecName: "kube-api-access-wll5d") pod "b73546bb-ca8e-42bb-81a5-615e93bf16b0" (UID: "b73546bb-ca8e-42bb-81a5-615e93bf16b0"). InnerVolumeSpecName "kube-api-access-wll5d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:09:28.747411 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.747340    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:28.747852 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.747833    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:28.747945 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.747855    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:28.748003 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.747968    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:28.751154 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.751134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:28.754396 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.754371    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:28.768460 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.766915    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wll5d\" (UniqueName: \"kubernetes.io/projected/b73546bb-ca8e-42bb-81a5-615e93bf16b0-kube-api-access-wll5d\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:09:28.868316 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.868272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttwtg\" (UniqueName: \"kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg\") pod \"test-curl-7843-139839173695792\" (UID: \"6c02f398-a67e-45a9-b4a5-5d8c705c5f99\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:28.968997 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.968958    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttwtg\" (UniqueName: \"kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg\") pod \"test-curl-7843-139839173695792\" (UID: \"6c02f398-a67e-45a9-b4a5-5d8c705c5f99\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:28.976994 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:28.976961    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttwtg\" (UniqueName: \"kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg\") pod \"test-curl-7843-139839173695792\" (UID: \"6c02f398-a67e-45a9-b4a5-5d8c705c5f99\") " pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:29.066941 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.066868    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:29.339348 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.339317    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:29.340179 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:29.340158    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6c02f398_a67e_45a9_b4a5_5d8c705c5f99.slice/crio-7b1f659d324bb519f76bddb0bc079d0e2acd49729f5a51a6e561a5856cc540eb WatchSource:0}: Error finding container 7b1f659d324bb519f76bddb0bc079d0e2acd49729f5a51a6e561a5856cc540eb: Status 404 returned error can't find the container with id 7b1f659d324bb519f76bddb0bc079d0e2acd49729f5a51a6e561a5856cc540eb
Sep 09 13:09:29.456968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.456946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:29.457096 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.456981    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1c5b366d3936184a978359930d23d11184cfe93a6ae753fa6cfeb80f9d06086c"
Sep 09 13:09:29.458185 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.458160    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695792" event={"ID":"6c02f398-a67e-45a9-b4a5-5d8c705c5f99","Type":"ContainerStarted","Data":"7b1f659d324bb519f76bddb0bc079d0e2acd49729f5a51a6e561a5856cc540eb"}
Sep 09 13:09:29.468431 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:29.468409    2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7843-139839173695792" oldPodUID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" podUID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99"
Sep 09 13:09:30.456628 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:30.456594    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b73546bb-ca8e-42bb-81a5-615e93bf16b0" path="/var/lib/kubelet/pods/b73546bb-ca8e-42bb-81a5-615e93bf16b0/volumes"
Sep 09 13:09:30.466433 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:30.466353    2580 generic.go:356] "Generic (PLEG): container finished" podID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99" containerID="7b77738200b999ce7f9838c29f1765840dd215aadb3a64a02f176ce3a7d36ecf" exitCode=0
Sep 09 13:09:30.466601 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:30.466441    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7843-139839173695792" event={"ID":"6c02f398-a67e-45a9-b4a5-5d8c705c5f99","Type":"ContainerDied","Data":"7b77738200b999ce7f9838c29f1765840dd215aadb3a64a02f176ce3a7d36ecf"}
Sep 09 13:09:30.483808 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:30.483776    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:30.873014 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:30.872914    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:31.616966 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.616943    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:31.634368 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.634342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7843-139839173695792_6c02f398-a67e-45a9-b4a5-5d8c705c5f99/test-curl-7843-139839173695792/0.log"
Sep 09 13:09:31.656839 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.656809    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:31.659145 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.659119    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7843-139839173695792"]
Sep 09 13:09:31.690214 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.690189    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ttwtg\" (UniqueName: \"kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg\") pod \"6c02f398-a67e-45a9-b4a5-5d8c705c5f99\" (UID: \"6c02f398-a67e-45a9-b4a5-5d8c705c5f99\") "
Sep 09 13:09:31.692240 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.692215    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg" (OuterVolumeSpecName: "kube-api-access-ttwtg") pod "6c02f398-a67e-45a9-b4a5-5d8c705c5f99" (UID: "6c02f398-a67e-45a9-b4a5-5d8c705c5f99"). InnerVolumeSpecName "kube-api-access-ttwtg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:09:31.790963 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:31.790905    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ttwtg\" (UniqueName: \"kubernetes.io/projected/6c02f398-a67e-45a9-b4a5-5d8c705c5f99-kube-api-access-ttwtg\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:09:32.314138 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.314100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:32.454978 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.454945    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99" path="/var/lib/kubelet/pods/6c02f398-a67e-45a9-b4a5-5d8c705c5f99/volumes"
Sep 09 13:09:32.480404 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.480380    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7843-139839173695792"
Sep 09 13:09:32.480545 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.480422    2580 scope.go:117] "RemoveContainer" containerID="7b77738200b999ce7f9838c29f1765840dd215aadb3a64a02f176ce3a7d36ecf"
Sep 09 13:09:32.647583 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.647547    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:32.882964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:32.882923    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:35.230207 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:35.230168    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:35.326733 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:35.326692    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:36.518911 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:36.518876    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:36.618997 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:36.618955    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:36.828652 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:36.828556    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:37.009345 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.009311    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:37.108465 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.108387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:37.114820 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.114788    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:09:37.115423 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.115405    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:37.115477 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.115427    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:37.115577 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.115565    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6c02f398-a67e-45a9-b4a5-5d8c705c5f99" containerName="test-curl-7843-139839173695792"
Sep 09 13:09:37.120435 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.120417    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.123361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.123326    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-597fffff\""
Sep 09 13:09:37.123361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.123359    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-597fffff-dockercfg-jx8cs\""
Sep 09 13:09:37.129557 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.129538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:09:37.247067 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.247037    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.247238 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.247141    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ngqkm\" (UniqueName: \"kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.348379 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.348349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ngqkm\" (UniqueName: \"kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.348556 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.348529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.349217 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.349191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.356353 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.356333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ngqkm\" (UniqueName: \"kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm\") pod \"payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.433112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.433086    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:37.715826 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:37.713402    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:09:37.717444 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:09:37.717412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7ea7add_5d12_42c5_a8d3_010d1b751aea.slice/crio-004e3b6b1c0d3516282eb73107044f9e2fccd498b695afed65f1d1804299da47 WatchSource:0}: Error finding container 004e3b6b1c0d3516282eb73107044f9e2fccd498b695afed65f1d1804299da47: Status 404 returned error can't find the container with id 004e3b6b1c0d3516282eb73107044f9e2fccd498b695afed65f1d1804299da47
Sep 09 13:09:38.523062 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:38.523026    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" event={"ID":"f7ea7add-5d12-42c5-a8d3-010d1b751aea","Type":"ContainerStarted","Data":"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"}
Sep 09 13:09:38.523262 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:38.523079    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" event={"ID":"f7ea7add-5d12-42c5-a8d3-010d1b751aea","Type":"ContainerStarted","Data":"004e3b6b1c0d3516282eb73107044f9e2fccd498b695afed65f1d1804299da47"}
Sep 09 13:09:38.523262 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:38.523156    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:38.540729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:38.540676    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" podStartSLOduration=1.540655804 podStartE2EDuration="1.540655804s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:09:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:09:38.536791909 +0000 UTC m=+1276.572039817" watchObservedRunningTime="2026-09-09 13:09:38.540655804 +0000 UTC m=+1276.575903715"
Sep 09 13:09:45.067721 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:45.067684    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:09:49.533302 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:49.533270    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:09:58.386981 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:09:58.386945    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:10:00.289758 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:00.289718    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:10:02.869442 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.869410    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:10:02.873855 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.873836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.877248 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.877221    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-597fffff\"/\"default-dockercfg-xtb9j\""
Sep 09 13:10:02.877377 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.877284    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-597fffff\"/\"kube-root-ca.crt\""
Sep 09 13:10:02.877377 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.877288    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-597fffff\"/\"openshift-service-ca.crt\""
Sep 09 13:10:02.877546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.877519    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-597fffff\"/\"e2e-lifecycle-model-597fffff-kserve-self-signed-certs\""
Sep 09 13:10:02.880516 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.880483    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:10:02.894429 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.894398    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:10:02.973029 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973002    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.973163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973050    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.973163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6z6jv\" (UniqueName: \"kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.973163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973114    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.973163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973151    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:02.973315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:02.973218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074168 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6z6jv\" (UniqueName: \"kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074230    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074245    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074636 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074694 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074645    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.074745 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.074721    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.076546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.076516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.076806 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.076790    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.081663 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.081641    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6z6jv\" (UniqueName: \"kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv\") pod \"e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") " pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.185700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.185671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:03.458783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.458757    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:10:03.459968 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:10:03.459940    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8a090608_b7da_406b_9109_c80ed712f47d.slice/crio-563db635e9bf143943a22733e72f3e092877f86888f26d8cf15464f6795d9c83 WatchSource:0}: Error finding container 563db635e9bf143943a22733e72f3e092877f86888f26d8cf15464f6795d9c83: Status 404 returned error can't find the container with id 563db635e9bf143943a22733e72f3e092877f86888f26d8cf15464f6795d9c83
Sep 09 13:10:03.682256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.682204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" event={"ID":"8a090608-b7da-406b-9109-c80ed712f47d","Type":"ContainerStarted","Data":"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"}
Sep 09 13:10:03.682256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.682259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" event={"ID":"8a090608-b7da-406b-9109-c80ed712f47d","Type":"ContainerStarted","Data":"563db635e9bf143943a22733e72f3e092877f86888f26d8cf15464f6795d9c83"}
Sep 09 13:10:03.697954 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:03.697899    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" podStartSLOduration=1.697883081 podStartE2EDuration="1.697883081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:10:03.696671167 +0000 UTC m=+1301.731919076" watchObservedRunningTime="2026-09-09 13:10:03.697883081 +0000 UTC m=+1301.733130987"
Sep 09 13:10:13.186045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:13.186007    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:13.186564 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:13.186067    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:13.198842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:13.198816    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:13.758058 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:13.758029    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:10:45.121729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.121641    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"]
Sep 09 13:10:45.126882 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.126859    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:45.133095 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.133066    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"]
Sep 09 13:10:45.133313 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.133282    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:10:45.238141 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.238108    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-597fffff-rcnb4\" (UID: \"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:45.339083 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.339050    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-597fffff-rcnb4\" (UID: \"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:45.339668 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.339648    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-597fffff-rcnb4\" (UID: \"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:45.441742 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.441718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:45.715010 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.714979    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"]
Sep 09 13:10:45.716840 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:10:45.716805    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4e848f_dd78_4e48_94a4_69cadb1c0ab4.slice/crio-187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85 WatchSource:0}: Error finding container 187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85: Status 404 returned error can't find the container with id 187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85
Sep 09 13:10:45.945764 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:45.945730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4" event={"ID":"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4","Type":"ContainerStarted","Data":"187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85"}
Sep 09 13:10:48.965468 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:48.965430    2580 generic.go:356] "Generic (PLEG): container finished" podID="fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" containerID="970326b88ffef4ebd48e9cc76d8e9cd8e31c7dad2f1b2cb65049d3c2bde85cec" exitCode=0
Sep 09 13:10:48.965858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:48.965528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4" event={"ID":"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4","Type":"ContainerDied","Data":"970326b88ffef4ebd48e9cc76d8e9cd8e31c7dad2f1b2cb65049d3c2bde85cec"}
Sep 09 13:10:50.111818 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.111795    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:50.285786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.285693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca\") pod \"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4\" (UID: \"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4\") "
Sep 09 13:10:50.286064 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.286041    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" (UID: "fc4e848f-dd78-4e48-94a4-69cadb1c0ab4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:10:50.386698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.386670    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:10:50.979616 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.979584    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4" event={"ID":"fc4e848f-dd78-4e48-94a4-69cadb1c0ab4","Type":"ContainerDied","Data":"187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85"}
Sep 09 13:10:50.979616 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.979614    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"
Sep 09 13:10:50.979927 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:10:50.979641    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="187a5f833e5004094d0fa45391b568c0dbc97d27f6a77b11153ed66ee489fc85"
Sep 09 13:11:00.617383 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.617342    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:00.651461 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.651427    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:11:00.651708 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.651683    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" podUID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" containerName="payload-pre-processing" containerID="cri-o://6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06" gracePeriod=30
Sep 09 13:11:00.717287 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.717248    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:00.807560 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.807495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"]
Sep 09 13:11:00.809171 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.809140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:00.811017 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.810987    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-597fffff-rcnb4"]
Sep 09 13:11:00.870007 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.869919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:00.909012 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.908990    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:11:00.984753 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.984723    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ngqkm\" (UniqueName: \"kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm\") pod \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") "
Sep 09 13:11:00.984923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.984807    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume\") pod \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\" (UID: \"f7ea7add-5d12-42c5-a8d3-010d1b751aea\") "
Sep 09 13:11:00.985117 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.985097    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7ea7add-5d12-42c5-a8d3-010d1b751aea" (UID: "f7ea7add-5d12-42c5-a8d3-010d1b751aea"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:11:00.986901 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:00.986882    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm" (OuterVolumeSpecName: "kube-api-access-ngqkm") pod "f7ea7add-5d12-42c5-a8d3-010d1b751aea" (UID: "f7ea7add-5d12-42c5-a8d3-010d1b751aea"). InnerVolumeSpecName "kube-api-access-ngqkm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:11:01.042644 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.042615    2580 generic.go:356] "Generic (PLEG): container finished" podID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" containerID="6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06" exitCode=0
Sep 09 13:11:01.042769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.042660    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"
Sep 09 13:11:01.042769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.042700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" event={"ID":"f7ea7add-5d12-42c5-a8d3-010d1b751aea","Type":"ContainerDied","Data":"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"}
Sep 09 13:11:01.042769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.042755    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk" event={"ID":"f7ea7add-5d12-42c5-a8d3-010d1b751aea","Type":"ContainerDied","Data":"004e3b6b1c0d3516282eb73107044f9e2fccd498b695afed65f1d1804299da47"}
Sep 09 13:11:01.042874 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.042777    2580 scope.go:117] "RemoveContainer" containerID="6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"
Sep 09 13:11:01.052207 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.052194    2580 scope.go:117] "RemoveContainer" containerID="6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"
Sep 09 13:11:01.052470 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:11:01.052446    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06\": container with ID starting with 6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06 not found: ID does not exist" containerID="6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"
Sep 09 13:11:01.052595 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.052476    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06"} err="failed to get container status \"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06\": rpc error: code = NotFound desc = could not find container \"6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06\": container with ID starting with 6ebc3aa2c460d30fe0cb246d8c94602c0cb257d4b0b7d5ee93624ae1770bdc06 not found: ID does not exist"
Sep 09 13:11:01.063653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.063633    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:11:01.069191 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.069169    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-597fffff-8676d89c66-jwgwk"]
Sep 09 13:11:01.086069 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.086050    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7ea7add-5d12-42c5-a8d3-010d1b751aea-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:01.086146 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:01.086071    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ngqkm\" (UniqueName: \"kubernetes.io/projected/f7ea7add-5d12-42c5-a8d3-010d1b751aea-kube-api-access-ngqkm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:02.454005 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:02.453968    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" path="/var/lib/kubelet/pods/f7ea7add-5d12-42c5-a8d3-010d1b751aea/volumes"
Sep 09 13:11:02.454385 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:02.454347    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" path="/var/lib/kubelet/pods/fc4e848f-dd78-4e48-94a4-69cadb1c0ab4/volumes"
Sep 09 13:11:05.700682 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:05.700639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:09.169045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:09.169010    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:11:09.406486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:09.406447    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:09.506740 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:09.506662    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:15.808028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:15.807991    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:24.189774 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.189713    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" podUID="8a090608-b7da-406b-9109-c80ed712f47d" containerName="main" containerID="cri-o://522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24" gracePeriod=45
Sep 09 13:11:24.445857 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.445794    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:11:24.610407 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610374    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610421    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610536    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6z6jv\" (UniqueName: \"kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610770 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610631    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610770 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610657    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache\") pod \"8a090608-b7da-406b-9109-c80ed712f47d\" (UID: \"8a090608-b7da-406b-9109-c80ed712f47d\") "
Sep 09 13:11:24.610770 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610721    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:24.610938 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610913    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home" (OuterVolumeSpecName: "home") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:24.611024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.610978    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:24.611081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.611037    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache" (OuterVolumeSpecName: "model-cache") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:24.612658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.612632    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm" (OuterVolumeSpecName: "dshm") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:24.612771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.612687    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:11:24.612771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.612711    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv" (OuterVolumeSpecName: "kube-api-access-6z6jv") pod "8a090608-b7da-406b-9109-c80ed712f47d" (UID: "8a090608-b7da-406b-9109-c80ed712f47d"). InnerVolumeSpecName "kube-api-access-6z6jv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:11:24.712370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.712286    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:24.712370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.712312    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:24.712370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.712322    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8a090608-b7da-406b-9109-c80ed712f47d-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:24.712370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.712329    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8a090608-b7da-406b-9109-c80ed712f47d-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:24.712370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:24.712338    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6z6jv\" (UniqueName: \"kubernetes.io/projected/8a090608-b7da-406b-9109-c80ed712f47d-kube-api-access-6z6jv\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:25.190542 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.190486    2580 generic.go:356] "Generic (PLEG): container finished" podID="8a090608-b7da-406b-9109-c80ed712f47d" containerID="522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24" exitCode=0
Sep 09 13:11:25.190964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.190537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" event={"ID":"8a090608-b7da-406b-9109-c80ed712f47d","Type":"ContainerDied","Data":"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"}
Sep 09 13:11:25.190964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.190590    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn" event={"ID":"8a090608-b7da-406b-9109-c80ed712f47d","Type":"ContainerDied","Data":"563db635e9bf143943a22733e72f3e092877f86888f26d8cf15464f6795d9c83"}
Sep 09 13:11:25.190964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.190608    2580 scope.go:117] "RemoveContainer" containerID="522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"
Sep 09 13:11:25.190964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.190551    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"
Sep 09 13:11:25.200921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.200901    2580 scope.go:117] "RemoveContainer" containerID="522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"
Sep 09 13:11:25.201200 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:11:25.201182    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24\": container with ID starting with 522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24 not found: ID does not exist" containerID="522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"
Sep 09 13:11:25.201263 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.201208    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24"} err="failed to get container status \"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24\": rpc error: code = NotFound desc = could not find container \"522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24\": container with ID starting with 522d05f3400d4614dc647436dfc94ed36fb17b2e1c4312d11eb807cb61b17f24 not found: ID does not exist"
Sep 09 13:11:25.212195 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.212174    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:11:25.217397 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:25.217371    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-597fffff/e2e-lifecycle-model-597fffff-kserve-56f96c884c-rhjqn"]
Sep 09 13:11:26.453394 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:26.453363    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a090608-b7da-406b-9109-c80ed712f47d" path="/var/lib/kubelet/pods/8a090608-b7da-406b-9109-c80ed712f47d/volumes"
Sep 09 13:11:30.241257 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241215    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7846-8358b6"]
Sep 09 13:11:30.241866 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a090608-b7da-406b-9109-c80ed712f47d" containerName="main"
Sep 09 13:11:30.241968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241868    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a090608-b7da-406b-9109-c80ed712f47d" containerName="main"
Sep 09 13:11:30.241968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241901    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" containerName="revoke-keys"
Sep 09 13:11:30.241968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241909    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" containerName="revoke-keys"
Sep 09 13:11:30.241968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241929    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" containerName="payload-pre-processing"
Sep 09 13:11:30.241968 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.241938    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" containerName="payload-pre-processing"
Sep 09 13:11:30.242233 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.242044    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a090608-b7da-406b-9109-c80ed712f47d" containerName="main"
Sep 09 13:11:30.242233 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.242062    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7ea7add-5d12-42c5-a8d3-010d1b751aea" containerName="payload-pre-processing"
Sep 09 13:11:30.242233 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.242073    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4e848f-dd78-4e48-94a4-69cadb1c0ab4" containerName="revoke-keys"
Sep 09 13:11:30.244945 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.244923    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:30.247043 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.247018    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s2rqh\""
Sep 09 13:11:30.251420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.251396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-8358b6"]
Sep 09 13:11:30.364160 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.364118    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdsvb\" (UniqueName: \"kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb\") pod \"mt-curl-7846-8358b6\" (UID: \"520228cb-e543-4c65-8cb3-84dfae0a1906\") " pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:30.465033 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.465002    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdsvb\" (UniqueName: \"kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb\") pod \"mt-curl-7846-8358b6\" (UID: \"520228cb-e543-4c65-8cb3-84dfae0a1906\") " pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:30.472335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.472309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdsvb\" (UniqueName: \"kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb\") pod \"mt-curl-7846-8358b6\" (UID: \"520228cb-e543-4c65-8cb3-84dfae0a1906\") " pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:30.555407 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.555308    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:30.829615 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:30.829588    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-8358b6"]
Sep 09 13:11:30.830833 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:11:30.830806    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod520228cb_e543_4c65_8cb3_84dfae0a1906.slice/crio-8496f643f85538f151244c45f508cc9ef1eb8dca4cae964af9978ec9b0242447 WatchSource:0}: Error finding container 8496f643f85538f151244c45f508cc9ef1eb8dca4cae964af9978ec9b0242447: Status 404 returned error can't find the container with id 8496f643f85538f151244c45f508cc9ef1eb8dca4cae964af9978ec9b0242447
Sep 09 13:11:31.234186 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:31.234138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-8358b6" event={"ID":"520228cb-e543-4c65-8cb3-84dfae0a1906","Type":"ContainerStarted","Data":"8496f643f85538f151244c45f508cc9ef1eb8dca4cae964af9978ec9b0242447"}
Sep 09 13:11:32.104717 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:32.104679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:32.158076 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:32.158039    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:32.241272 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:32.241240    2580 generic.go:356] "Generic (PLEG): container finished" podID="520228cb-e543-4c65-8cb3-84dfae0a1906" containerID="423272c650007a48755c3ceeeceec9cfbb5017c900ed63e0eefd933da7f51db0" exitCode=0
Sep 09 13:11:32.241463 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:32.241375    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-8358b6" event={"ID":"520228cb-e543-4c65-8cb3-84dfae0a1906","Type":"ContainerDied","Data":"423272c650007a48755c3ceeeceec9cfbb5017c900ed63e0eefd933da7f51db0"}
Sep 09 13:11:33.387008 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.386983    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:33.393175 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.393150    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdsvb\" (UniqueName: \"kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb\") pod \"520228cb-e543-4c65-8cb3-84dfae0a1906\" (UID: \"520228cb-e543-4c65-8cb3-84dfae0a1906\") "
Sep 09 13:11:33.395010 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.394984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb" (OuterVolumeSpecName: "kube-api-access-tdsvb") pod "520228cb-e543-4c65-8cb3-84dfae0a1906" (UID: "520228cb-e543-4c65-8cb3-84dfae0a1906"). InnerVolumeSpecName "kube-api-access-tdsvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:11:33.406076 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.406031    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7846-8358b6_520228cb-e543-4c65-8cb3-84dfae0a1906/mt-curl-7846-8358b6/0.log"
Sep 09 13:11:33.427857 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.427834    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7846-8358b6"]
Sep 09 13:11:33.430196 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.430175    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7846-8358b6"]
Sep 09 13:11:33.493850 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:33.493824    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdsvb\" (UniqueName: \"kubernetes.io/projected/520228cb-e543-4c65-8cb3-84dfae0a1906-kube-api-access-tdsvb\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:34.260899 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:34.260868    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:34.260899 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:34.260893    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8496f643f85538f151244c45f508cc9ef1eb8dca4cae964af9978ec9b0242447"
Sep 09 13:11:34.270536 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:11:34.270494    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7846-8358b6\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="520228cb-e543-4c65-8cb3-84dfae0a1906" pod="opendatahub/mt-curl-7846-8358b6"
Sep 09 13:11:34.454911 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:34.454875    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="520228cb-e543-4c65-8cb3-84dfae0a1906" path="/var/lib/kubelet/pods/520228cb-e543-4c65-8cb3-84dfae0a1906/volumes"
Sep 09 13:11:34.810286 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:34.810230    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:11:34.828625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:34.828593    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:35.864335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:35.864305    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:46.201352 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:46.201317    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:46.309885 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:46.309844    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:49.832904 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:49.832847    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" podUID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" containerName="main" containerID="cri-o://fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9" gracePeriod=45
Sep 09 13:11:50.087297 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.087239    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:11:50.138161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138173    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138231    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138254    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f4wlr\" (UniqueName: \"kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138281    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138305    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm\") pod \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\" (UID: \"2612ef6e-2762-4236-b5e0-b8e1c4b3d894\") "
Sep 09 13:11:50.138546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138374    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache" (OuterVolumeSpecName: "model-cache") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:50.138546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138529    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:50.138638 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138566    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home" (OuterVolumeSpecName: "home") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:50.138638 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138631    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.138718 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138646    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.138718 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.138656    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.140548 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.140520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm" (OuterVolumeSpecName: "dshm") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:11:50.140646 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.140574    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:11:50.140646 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.140616    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr" (OuterVolumeSpecName: "kube-api-access-f4wlr") pod "2612ef6e-2762-4236-b5e0-b8e1c4b3d894" (UID: "2612ef6e-2762-4236-b5e0-b8e1c4b3d894"). InnerVolumeSpecName "kube-api-access-f4wlr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:11:50.239663 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.239634    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.239663 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.239659    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.239663 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.239668    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f4wlr\" (UniqueName: \"kubernetes.io/projected/2612ef6e-2762-4236-b5e0-b8e1c4b3d894-kube-api-access-f4wlr\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:11:50.365300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.365217    2580 generic.go:356] "Generic (PLEG): container finished" podID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" containerID="fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9" exitCode=0
Sep 09 13:11:50.365300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.365263    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"
Sep 09 13:11:50.365527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.365305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" event={"ID":"2612ef6e-2762-4236-b5e0-b8e1c4b3d894","Type":"ContainerDied","Data":"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"}
Sep 09 13:11:50.365527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.365347    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9" event={"ID":"2612ef6e-2762-4236-b5e0-b8e1c4b3d894","Type":"ContainerDied","Data":"da22fb647123cd8de9d178f57bfb259b5023b8af8d82813e0c666978b53d107d"}
Sep 09 13:11:50.365527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.365364    2580 scope.go:117] "RemoveContainer" containerID="fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"
Sep 09 13:11:50.375385 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.375369    2580 scope.go:117] "RemoveContainer" containerID="fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"
Sep 09 13:11:50.375702 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:11:50.375684    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9\": container with ID starting with fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9 not found: ID does not exist" containerID="fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"
Sep 09 13:11:50.375769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.375709    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9"} err="failed to get container status \"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9\": rpc error: code = NotFound desc = could not find container \"fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9\": container with ID starting with fc0c73de280f38c63e2ca6523357698ef3e868c060f74350566881e88b943af9 not found: ID does not exist"
Sep 09 13:11:50.387613 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.387590    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:11:50.391268 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.391248    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/auth-test-model-ada1ae-kserve-9465b64f5-dq6z9"]
Sep 09 13:11:50.452757 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:50.452736    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" path="/var/lib/kubelet/pods/2612ef6e-2762-4236-b5e0-b8e1c4b3d894/volumes"
Sep 09 13:11:51.780001 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:51.779959    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:56.173108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.173072    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:11:56.173998 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.173966    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" containerName="main"
Sep 09 13:11:56.173998 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.173999    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" containerName="main"
Sep 09 13:11:56.173998 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.174010    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="520228cb-e543-4c65-8cb3-84dfae0a1906" containerName="mt-curl-7846-8358b6"
Sep 09 13:11:56.174203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.174016    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="520228cb-e543-4c65-8cb3-84dfae0a1906" containerName="mt-curl-7846-8358b6"
Sep 09 13:11:56.174203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.174091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="520228cb-e543-4c65-8cb3-84dfae0a1906" containerName="mt-curl-7846-8358b6"
Sep 09 13:11:56.174203 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.174101    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="2612ef6e-2762-4236-b5e0-b8e1c4b3d894" containerName="main"
Sep 09 13:11:56.177256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.177233    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.180312 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.180287    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"openshift-service-ca.crt\""
Sep 09 13:11:56.180439 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.180312    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"sub-test-model-ada1ae-kserve-self-signed-certs\""
Sep 09 13:11:56.180439 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.180342    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"kube-root-ca.crt\""
Sep 09 13:11:56.180439 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.180355    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"default-dockercfg-7xt76\""
Sep 09 13:11:56.190734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.190713    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:11:56.203314 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.203267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:11:56.291994 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.291969    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.292158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.292006    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.292158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.292109    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hxvp\" (UniqueName: \"kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.292158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.292148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.292308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.292189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.292308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.292239    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392585 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392559    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392603    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392626    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392678    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hxvp\" (UniqueName: \"kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392698    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392720    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.392995 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392965    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.393105 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.392973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.393105 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.393062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.394892 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.394872    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.395134 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.395115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.400253 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.400231    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hxvp\" (UniqueName: \"kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp\") pod \"sub-test-model-ada1ae-kserve-65665f46b9-rw878\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.490285 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.490189    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:11:56.765156 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:56.765131    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:11:56.766900 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:11:56.766874    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod581f7ade_e515_4839_92e3_93a180630bda.slice/crio-93e2e17ac7d70bc22d63c033d19b8b302d0f808e78399adedea9a8485eaccfb6 WatchSource:0}: Error finding container 93e2e17ac7d70bc22d63c033d19b8b302d0f808e78399adedea9a8485eaccfb6: Status 404 returned error can't find the container with id 93e2e17ac7d70bc22d63c033d19b8b302d0f808e78399adedea9a8485eaccfb6
Sep 09 13:11:57.414909 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:57.414874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" event={"ID":"581f7ade-e515-4839-92e3-93a180630bda","Type":"ContainerStarted","Data":"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"}
Sep 09 13:11:57.415380 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:57.414917    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" event={"ID":"581f7ade-e515-4839-92e3-93a180630bda","Type":"ContainerStarted","Data":"93e2e17ac7d70bc22d63c033d19b8b302d0f808e78399adedea9a8485eaccfb6"}
Sep 09 13:11:57.431698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:11:57.431646    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" podStartSLOduration=1.431632693 podStartE2EDuration="1.431632693s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:11:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:11:57.428950511 +0000 UTC m=+1415.464198422" watchObservedRunningTime="2026-09-09 13:11:57.431632693 +0000 UTC m=+1415.466880642"
Sep 09 13:12:06.036694 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.036659    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:06.137597 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.137564    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:06.337026 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.336942    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:06.491069 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.491039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:12:06.491235 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.491082    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:12:06.503521 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:06.503488    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:12:07.486267 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:07.486242    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:12:07.702088 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:07.702047    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:07.861905 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:07.861800    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:08.486139 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:08.486109    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:08.647963 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:08.647922    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:08.748479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:08.748386    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:10.067781 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:10.067742    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:15.749004 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:15.748891    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:15.854200 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:15.854170    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:16.450715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:16.450676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:17.966841 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.966797    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:12:17.973392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.973367    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:17.976286 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.976262    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"kube-root-ca.crt\""
Sep 09 13:12:17.977030 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.977005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"sub-test-model-650a99-kserve-self-signed-certs\""
Sep 09 13:12:17.977163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.977048    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"openshift-service-ca.crt\""
Sep 09 13:12:17.977163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.977005    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"default-dockercfg-n72nf\""
Sep 09 13:12:17.979715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.979689    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:12:17.995832 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:17.995797    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:18.089300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.089300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.089527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089327    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.089527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgjnd\" (UniqueName: \"kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.089527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.089527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.089418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190174    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190259    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190542 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190334    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgjnd\" (UniqueName: \"kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190628 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190758 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.190758 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.190731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.192488 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.192465    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.192695 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.192678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.198760 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.198737    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgjnd\" (UniqueName: \"kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd\") pod \"sub-test-model-650a99-kserve-87bf58f95-2wntl\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") " pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.286479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.286409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:18.565917 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:18.565886    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:12:18.566607 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:12:18.566584    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc54a2b87_e7b2_4284_a9dc_9beb86aa8b83.slice/crio-b1dc24436c089965f11680981e40140243941d41c7e2f6d08f43c55d35d3993f WatchSource:0}: Error finding container b1dc24436c089965f11680981e40140243941d41c7e2f6d08f43c55d35d3993f: Status 404 returned error can't find the container with id b1dc24436c089965f11680981e40140243941d41c7e2f6d08f43c55d35d3993f
Sep 09 13:12:19.546164 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:19.546126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" event={"ID":"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83","Type":"ContainerStarted","Data":"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"}
Sep 09 13:12:19.546164 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:19.546170    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" event={"ID":"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83","Type":"ContainerStarted","Data":"b1dc24436c089965f11680981e40140243941d41c7e2f6d08f43c55d35d3993f"}
Sep 09 13:12:19.564844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:19.564800    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" podStartSLOduration=2.564786185 podStartE2EDuration="2.564786185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:12:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:12:19.562842786 +0000 UTC m=+1437.598090706" watchObservedRunningTime="2026-09-09 13:12:19.564786185 +0000 UTC m=+1437.600034092"
Sep 09 13:12:22.336662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:22.336622    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:22.453579 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:22.453546    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:28.286849 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:28.286810    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:28.286849 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:28.286854    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:28.299562 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:28.299539    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:28.619811 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:28.619727    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:12:33.037916 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:33.037878    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:33.241132 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:33.241097    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:12:54.282193 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:12:54.282150    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:00.114143 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:00.114093    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:00.173103 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:00.173068    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:00.501495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:00.501453    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:00.931394 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:00.931362    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:01.121201 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:01.121160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:03.328853 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.328817    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7846-7c61af"]
Sep 09 13:13:03.332861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.332836    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:03.335205 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.335181    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-s2rqh\""
Sep 09 13:13:03.336700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.336674    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-7c61af"]
Sep 09 13:13:03.407548 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.407496    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tnvxf\" (UniqueName: \"kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf\") pod \"mt-curl-7846-7c61af\" (UID: \"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a\") " pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:03.508093 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.508052    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tnvxf\" (UniqueName: \"kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf\") pod \"mt-curl-7846-7c61af\" (UID: \"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a\") " pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:03.515988 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.515952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tnvxf\" (UniqueName: \"kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf\") pod \"mt-curl-7846-7c61af\" (UID: \"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a\") " pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:03.644711 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.644663    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:03.921200 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:03.921173    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-7c61af"]
Sep 09 13:13:03.922702 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:13:03.922674    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podec2f1e1f_9e98_4d55_8f2d_07ee1670f75a.slice/crio-356ea16abe9323c6e8fe70491e4dd8ec37ef37a1053b6d0afc53fa7f07bc4582 WatchSource:0}: Error finding container 356ea16abe9323c6e8fe70491e4dd8ec37ef37a1053b6d0afc53fa7f07bc4582: Status 404 returned error can't find the container with id 356ea16abe9323c6e8fe70491e4dd8ec37ef37a1053b6d0afc53fa7f07bc4582
Sep 09 13:13:04.503279 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:04.503195    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:04.603161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:04.603125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:04.837971 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:04.837878    2580 generic.go:356] "Generic (PLEG): container finished" podID="ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" containerID="c8d47b35eed38462063f1fd9fe6e9a1cf2de0457515f774eaf4479fb24594888" exitCode=0
Sep 09 13:13:04.838121 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:04.837970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-7c61af" event={"ID":"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a","Type":"ContainerDied","Data":"c8d47b35eed38462063f1fd9fe6e9a1cf2de0457515f774eaf4479fb24594888"}
Sep 09 13:13:04.838121 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:04.838020    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-7c61af" event={"ID":"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a","Type":"ContainerStarted","Data":"356ea16abe9323c6e8fe70491e4dd8ec37ef37a1053b6d0afc53fa7f07bc4582"}
Sep 09 13:13:05.978763 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:05.978738    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:05.998164 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:05.998135    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7846-7c61af_ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a/mt-curl-7846-7c61af/0.log"
Sep 09 13:13:06.020880 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.020848    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7846-7c61af"]
Sep 09 13:13:06.023245 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.023218    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7846-7c61af"]
Sep 09 13:13:06.033380 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.033357    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tnvxf\" (UniqueName: \"kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf\") pod \"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a\" (UID: \"ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a\") "
Sep 09 13:13:06.035373 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.035353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf" (OuterVolumeSpecName: "kube-api-access-tnvxf") pod "ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" (UID: "ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a"). InnerVolumeSpecName "kube-api-access-tnvxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:13:06.135038 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.134960    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tnvxf\" (UniqueName: \"kubernetes.io/projected/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a-kube-api-access-tnvxf\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:06.166081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.166052    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7846-43840e"]
Sep 09 13:13:06.166517 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.166487    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" containerName="mt-curl-7846-7c61af"
Sep 09 13:13:06.166517 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.166515    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" containerName="mt-curl-7846-7c61af"
Sep 09 13:13:06.166629 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.166595    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" containerName="mt-curl-7846-7c61af"
Sep 09 13:13:06.169784 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.169769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:06.176269 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.176231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-43840e"]
Sep 09 13:13:06.236449 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.236413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4b8f\" (UniqueName: \"kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f\") pod \"mt-curl-7846-43840e\" (UID: \"3893cc7b-5c83-4d92-af68-4435842f2a7f\") " pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:06.337794 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.337754    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4b8f\" (UniqueName: \"kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f\") pod \"mt-curl-7846-43840e\" (UID: \"3893cc7b-5c83-4d92-af68-4435842f2a7f\") " pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:06.344909 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.344883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4b8f\" (UniqueName: \"kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f\") pod \"mt-curl-7846-43840e\" (UID: \"3893cc7b-5c83-4d92-af68-4435842f2a7f\") " pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:06.453342 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.453310    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a" path="/var/lib/kubelet/pods/ec2f1e1f-9e98-4d55-8f2d-07ee1670f75a/volumes"
Sep 09 13:13:06.481126 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.481102    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:06.695388 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.695348    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:06.754351 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.754318    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7846-43840e"]
Sep 09 13:13:06.754610 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.754589    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:06.755473 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:13:06.755453    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3893cc7b_5c83_4d92_af68_4435842f2a7f.slice/crio-a3c008b371ada28b68173176a7a455b693fae6ced915b303ef00bdde07226d76 WatchSource:0}: Error finding container a3c008b371ada28b68173176a7a455b693fae6ced915b303ef00bdde07226d76: Status 404 returned error can't find the container with id a3c008b371ada28b68173176a7a455b693fae6ced915b303ef00bdde07226d76
Sep 09 13:13:06.851370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.851345    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-7c61af"
Sep 09 13:13:06.851525 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.851388    2580 scope.go:117] "RemoveContainer" containerID="c8d47b35eed38462063f1fd9fe6e9a1cf2de0457515f774eaf4479fb24594888"
Sep 09 13:13:06.852844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:06.852817    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-43840e" event={"ID":"3893cc7b-5c83-4d92-af68-4435842f2a7f","Type":"ContainerStarted","Data":"a3c008b371ada28b68173176a7a455b693fae6ced915b303ef00bdde07226d76"}
Sep 09 13:13:07.861131 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:07.861098    2580 generic.go:356] "Generic (PLEG): container finished" podID="3893cc7b-5c83-4d92-af68-4435842f2a7f" containerID="7b7f557c0f557a002dc63e1dc4f3de1130ba2a7247f67652323c0e9639ae08bc" exitCode=0
Sep 09 13:13:07.861131 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:07.861134    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7846-43840e" event={"ID":"3893cc7b-5c83-4d92-af68-4435842f2a7f","Type":"ContainerDied","Data":"7b7f557c0f557a002dc63e1dc4f3de1130ba2a7247f67652323c0e9639ae08bc"}
Sep 09 13:13:09.000346 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.000319    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:09.018867 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.018831    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7846-43840e_3893cc7b-5c83-4d92-af68-4435842f2a7f/mt-curl-7846-43840e/0.log"
Sep 09 13:13:09.041079 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.041054    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7846-43840e"]
Sep 09 13:13:09.044328 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.044302    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7846-43840e"]
Sep 09 13:13:09.063349 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.063324    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4b8f\" (UniqueName: \"kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f\") pod \"3893cc7b-5c83-4d92-af68-4435842f2a7f\" (UID: \"3893cc7b-5c83-4d92-af68-4435842f2a7f\") "
Sep 09 13:13:09.065334 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.065310    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f" (OuterVolumeSpecName: "kube-api-access-x4b8f") pod "3893cc7b-5c83-4d92-af68-4435842f2a7f" (UID: "3893cc7b-5c83-4d92-af68-4435842f2a7f"). InnerVolumeSpecName "kube-api-access-x4b8f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:13:09.164115 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.164090    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4b8f\" (UniqueName: \"kubernetes.io/projected/3893cc7b-5c83-4d92-af68-4435842f2a7f-kube-api-access-x4b8f\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:09.874959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.874925    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:09.875131 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:09.874979    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a3c008b371ada28b68173176a7a455b693fae6ced915b303ef00bdde07226d76"
Sep 09 13:13:09.884472 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:13:09.884446    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7846-43840e\" is forbidden: User \"system:node:ip-10-0-137-1.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-137-1.ec2.internal' and this object" podUID="3893cc7b-5c83-4d92-af68-4435842f2a7f" pod="opendatahub/mt-curl-7846-43840e"
Sep 09 13:13:10.453486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:10.453445    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3893cc7b-5c83-4d92-af68-4435842f2a7f" path="/var/lib/kubelet/pods/3893cc7b-5c83-4d92-af68-4435842f2a7f/volumes"
Sep 09 13:13:11.232422 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:11.232393    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:13:11.250780 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:11.250746    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:12.174560 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:12.174527    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:13:12.191376 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:12.191299    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:15.471007 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.470964    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:13:15.471712 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.471690    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3893cc7b-5c83-4d92-af68-4435842f2a7f" containerName="mt-curl-7846-43840e"
Sep 09 13:13:15.471783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.471715    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3893cc7b-5c83-4d92-af68-4435842f2a7f" containerName="mt-curl-7846-43840e"
Sep 09 13:13:15.471827 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.471811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3893cc7b-5c83-4d92-af68-4435842f2a7f" containerName="mt-curl-7846-43840e"
Sep 09 13:13:15.475886 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.475866    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.478277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.478251    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-ada1ae\"/\"rate-test-model-ada1ae-kserve-self-signed-certs\""
Sep 09 13:13:15.483406 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.483388    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:13:15.519024 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.518987    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.519212 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.519044    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.519212 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.519071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.519212 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.519094    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.519384 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.519261    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4sms\" (UniqueName: \"kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.519384 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.519351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.561395 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.561351    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:15.620861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.620824    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.620880    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.620911    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.620939    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.621009    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4sms\" (UniqueName: \"kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621279 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.621074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621341 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.621316    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621528 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.621480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.621744 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.621618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.623600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.623576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.623892 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.623875    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.628264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.628244    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4sms\" (UniqueName: \"kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms\") pod \"rate-test-model-ada1ae-kserve-799f69769f-bnm2n\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") " pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:15.788050 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:15.787985    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:16.059017 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:16.058990    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:13:16.060479 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:13:16.060456    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e623355_2b1f_45a9_89a0_fa934408f0a7.slice/crio-a660b6c36199f3686bcca1e8dba4662450a89242fee63803b10f7e93c59c7bb4 WatchSource:0}: Error finding container a660b6c36199f3686bcca1e8dba4662450a89242fee63803b10f7e93c59c7bb4: Status 404 returned error can't find the container with id a660b6c36199f3686bcca1e8dba4662450a89242fee63803b10f7e93c59c7bb4
Sep 09 13:13:16.928018 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:16.927978    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" event={"ID":"7e623355-2b1f-45a9-89a0-fa934408f0a7","Type":"ContainerStarted","Data":"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"}
Sep 09 13:13:16.928402 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:16.928025    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" event={"ID":"7e623355-2b1f-45a9-89a0-fa934408f0a7","Type":"ContainerStarted","Data":"a660b6c36199f3686bcca1e8dba4662450a89242fee63803b10f7e93c59c7bb4"}
Sep 09 13:13:16.944214 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:16.944172    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" podStartSLOduration=1.944156843 podStartE2EDuration="1.944156843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:16.942872963 +0000 UTC m=+1494.978120872" watchObservedRunningTime="2026-09-09 13:13:16.944156843 +0000 UTC m=+1494.979404760"
Sep 09 13:13:22.560163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:22.560135    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:13:22.563154 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:22.563130    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:13:22.698800 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:22.698773    2580 scope.go:117] "RemoveContainer" containerID="397421f3cb0616f1623fd4cb6d611652e3ec94b804d249883ef097d9d922fbc9"
Sep 09 13:13:25.788598 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:25.788564    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:25.789147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:25.788765    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:25.801163 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:25.801143    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:25.994301 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:25.994275    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:13:26.252154 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.252114    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" podUID="581f7ade-e515-4839-92e3-93a180630bda" containerName="main" containerID="cri-o://fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47" gracePeriod=45
Sep 09 13:13:26.513357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.513291    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:13:26.616929 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.616896    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.616941    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.616980    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617015    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617051    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617102    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hxvp\" (UniqueName: \"kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp\") pod \"581f7ade-e515-4839-92e3-93a180630bda\" (UID: \"581f7ade-e515-4839-92e3-93a180630bda\") "
Sep 09 13:13:26.617392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home" (OuterVolumeSpecName: "home") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:26.617392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache" (OuterVolumeSpecName: "model-cache") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:26.617392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617210    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:26.617568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617408    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.617568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617422    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.617568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.617436    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.619102 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.619073    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm" (OuterVolumeSpecName: "dshm") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:26.619227 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.619120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:13:26.619227 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.619210    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp" (OuterVolumeSpecName: "kube-api-access-2hxvp") pod "581f7ade-e515-4839-92e3-93a180630bda" (UID: "581f7ade-e515-4839-92e3-93a180630bda"). InnerVolumeSpecName "kube-api-access-2hxvp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:13:26.718832 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.718796    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/581f7ade-e515-4839-92e3-93a180630bda-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.718832 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.718833    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/581f7ade-e515-4839-92e3-93a180630bda-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.718989 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.718848    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hxvp\" (UniqueName: \"kubernetes.io/projected/581f7ade-e515-4839-92e3-93a180630bda-kube-api-access-2hxvp\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:26.990018 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.989977    2580 generic.go:356] "Generic (PLEG): container finished" podID="581f7ade-e515-4839-92e3-93a180630bda" containerID="fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47" exitCode=0
Sep 09 13:13:26.990486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.990035    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"
Sep 09 13:13:26.990486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.990053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" event={"ID":"581f7ade-e515-4839-92e3-93a180630bda","Type":"ContainerDied","Data":"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"}
Sep 09 13:13:26.990486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.990100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878" event={"ID":"581f7ade-e515-4839-92e3-93a180630bda","Type":"ContainerDied","Data":"93e2e17ac7d70bc22d63c033d19b8b302d0f808e78399adedea9a8485eaccfb6"}
Sep 09 13:13:26.990486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.990117    2580 scope.go:117] "RemoveContainer" containerID="fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"
Sep 09 13:13:26.999543 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.999525    2580 scope.go:117] "RemoveContainer" containerID="fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"
Sep 09 13:13:26.999797 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:13:26.999778    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47\": container with ID starting with fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47 not found: ID does not exist" containerID="fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"
Sep 09 13:13:26.999895 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:26.999804    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47"} err="failed to get container status \"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47\": rpc error: code = NotFound desc = could not find container \"fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47\": container with ID starting with fa67ec25eeaa9bb08186995e3c7403d450fe09d006c086288981096300a59c47 not found: ID does not exist"
Sep 09 13:13:27.012038 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.012016    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:13:27.015958 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.015937    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/sub-test-model-ada1ae-kserve-65665f46b9-rw878"]
Sep 09 13:13:27.194945 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.194912    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" podUID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" containerName="main" containerID="cri-o://00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6" gracePeriod=45
Sep 09 13:13:27.460686 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.460662    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:13:27.526361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526278    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kgjnd\" (UniqueName: \"kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526323    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526361 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526351    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526367    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526439    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir\") pod \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\" (UID: \"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83\") "
Sep 09 13:13:27.526815 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526693    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache" (OuterVolumeSpecName: "model-cache") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:27.526892 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.526860    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:27.527045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.527021    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.527216 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.527033    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home" (OuterVolumeSpecName: "home") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:27.528749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.528726    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm" (OuterVolumeSpecName: "dshm") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:13:27.528912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.528892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:13:27.528983 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.528897    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd" (OuterVolumeSpecName: "kube-api-access-kgjnd") pod "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" (UID: "c54a2b87-e7b2-4284-a9dc-9beb86aa8b83"). InnerVolumeSpecName "kube-api-access-kgjnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:13:27.628215 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.628183    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kgjnd\" (UniqueName: \"kubernetes.io/projected/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-kube-api-access-kgjnd\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.628215 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.628207    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.628215 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.628218    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.628450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.628227    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.628450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.628234    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:13:27.998550 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.998491    2580 generic.go:356] "Generic (PLEG): container finished" podID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" containerID="00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6" exitCode=0
Sep 09 13:13:27.998550 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.998552    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"
Sep 09 13:13:27.999077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.998574    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" event={"ID":"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83","Type":"ContainerDied","Data":"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"}
Sep 09 13:13:27.999077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.998615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl" event={"ID":"c54a2b87-e7b2-4284-a9dc-9beb86aa8b83","Type":"ContainerDied","Data":"b1dc24436c089965f11680981e40140243941d41c7e2f6d08f43c55d35d3993f"}
Sep 09 13:13:27.999077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:27.998634    2580 scope.go:117] "RemoveContainer" containerID="00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"
Sep 09 13:13:28.008641 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.008623    2580 scope.go:117] "RemoveContainer" containerID="00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"
Sep 09 13:13:28.008911 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:13:28.008892    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6\": container with ID starting with 00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6 not found: ID does not exist" containerID="00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"
Sep 09 13:13:28.008981 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.008931    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6"} err="failed to get container status \"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6\": rpc error: code = NotFound desc = could not find container \"00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6\": container with ID starting with 00c4f3c58bc76b7a8d499fdd64c10fe63e6b869398b0eef2d74c155abd0f29a6 not found: ID does not exist"
Sep 09 13:13:28.021147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.021121    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:13:28.024956 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.024934    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/sub-test-model-650a99-kserve-87bf58f95-2wntl"]
Sep 09 13:13:28.453383 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.453342    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="581f7ade-e515-4839-92e3-93a180630bda" path="/var/lib/kubelet/pods/581f7ade-e515-4839-92e3-93a180630bda/volumes"
Sep 09 13:13:28.453849 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:28.453827    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" path="/var/lib/kubelet/pods/c54a2b87-e7b2-4284-a9dc-9beb86aa8b83/volumes"
Sep 09 13:13:29.664364 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:29.664331    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:30.138757 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.138664    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:30.431601 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.431569    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:13:30.432462 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432437    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" containerName="main"
Sep 09 13:13:30.432647 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432627    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" containerName="main"
Sep 09 13:13:30.432768 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432658    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="581f7ade-e515-4839-92e3-93a180630bda" containerName="main"
Sep 09 13:13:30.432768 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432667    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="581f7ade-e515-4839-92e3-93a180630bda" containerName="main"
Sep 09 13:13:30.432853 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432782    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="581f7ade-e515-4839-92e3-93a180630bda" containerName="main"
Sep 09 13:13:30.432853 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.432799    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c54a2b87-e7b2-4284-a9dc-9beb86aa8b83" containerName="main"
Sep 09 13:13:30.437727 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.437711    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.439854 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.439832    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 13:13:30.439973 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.439892    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-a813cb-openshift-default-dockercfg-h42l7\""
Sep 09 13:13:30.454862 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.454835    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:30.456113 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.456093    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:13:30.553098 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553058    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553304 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553135    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553304 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553304 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553251    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553304 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553390    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.553495 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.553409    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqcc4\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.555701 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.555667    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:30.654077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654046    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654109    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654176    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqcc4\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654286    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654496    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654703 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654545    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654867    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.654912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.654901    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.655229 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.655206    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.656754 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.656728    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.656970 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.656951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.661864 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.661837    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.662235 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.662218    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqcc4\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4\") pod \"e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") " pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:30.749706 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:30.749644    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:31.025289 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:31.025258    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:13:31.031650 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:31.031616    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:31.031783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:31.031685    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:31.031783 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:31.031714    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:32.027302 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:32.027262    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerStarted","Data":"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"}
Sep 09 13:13:32.027806 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:32.027349    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerStarted","Data":"a350390a0ae4c75c7f81853c986fd775dd9bc7afe7243d8abebb92ebd2d8d86f"}
Sep 09 13:13:32.047490 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:32.047447    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podStartSLOduration=2.047431733 podStartE2EDuration="2.047431733s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:32.044523551 +0000 UTC m=+1510.079771459" watchObservedRunningTime="2026-09-09 13:13:32.047431733 +0000 UTC m=+1510.082679639"
Sep 09 13:13:32.750112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:32.750076    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:32.755359 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:32.755335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:33.033152 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:33.033066    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:33.033872 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:33.033854    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:34.060954 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:34.060919    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:34.166596 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:34.166558    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:34.523677 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:34.523635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:35.246176 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:35.246140    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:35.347262 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:35.347228    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:35.472874 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:35.472842    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:35.733846 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:35.733810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:35.801720 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:35.801686    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:36.015281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.015198    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:13:36.019680 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.019469    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.021653 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.021628    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-a813cb-dockercfg-6s84v\""
Sep 09 13:13:36.021816 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.021667    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-a813cb\""
Sep 09 13:13:36.027007 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.026982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:13:36.105321 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.105292    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nj5nq\" (UniqueName: \"kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.105466 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.105340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.205820 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.205796    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nj5nq\" (UniqueName: \"kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.205941 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.205836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.206464 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.206445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.215420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.215396    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nj5nq\" (UniqueName: \"kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq\") pod \"payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") " pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.334909 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.334841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:36.608972 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:36.608943    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:13:36.611062 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:13:36.611037    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod813eb172_0739_4a92_b0ce_2cbd34357cb5.slice/crio-732db7f8ec555c5f35f3853e452997cec61d5b4e90265d8a21dd6549e2d7daa0 WatchSource:0}: Error finding container 732db7f8ec555c5f35f3853e452997cec61d5b4e90265d8a21dd6549e2d7daa0: Status 404 returned error can't find the container with id 732db7f8ec555c5f35f3853e452997cec61d5b4e90265d8a21dd6549e2d7daa0
Sep 09 13:13:37.063089 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:37.063046    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" event={"ID":"813eb172-0739-4a92-b0ce-2cbd34357cb5","Type":"ContainerStarted","Data":"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"}
Sep 09 13:13:37.063292 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:37.063100    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" event={"ID":"813eb172-0739-4a92-b0ce-2cbd34357cb5","Type":"ContainerStarted","Data":"732db7f8ec555c5f35f3853e452997cec61d5b4e90265d8a21dd6549e2d7daa0"}
Sep 09 13:13:37.063292 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:37.063187    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:37.079161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:37.079119    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" podStartSLOduration=2.079107503 podStartE2EDuration="2.079107503s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:37.077830261 +0000 UTC m=+1515.113078179" watchObservedRunningTime="2026-09-09 13:13:37.079107503 +0000 UTC m=+1515.114355409"
Sep 09 13:13:45.540108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:45.540043    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:47.479692 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:47.479660    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:48.071309 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:48.071276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:13:52.457600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.457569    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:13:52.464460 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.464438    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.466773 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.466749    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"kube-root-ca.crt\""
Sep 09 13:13:52.467148 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.467131    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"openshift-service-ca.crt\""
Sep 09 13:13:52.468328 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.468077    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"default-dockercfg-n72nf\""
Sep 09 13:13:52.468328 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.468184    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-650a99\"/\"rate-test-model-650a99-kserve-self-signed-certs\""
Sep 09 13:13:52.471657 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.471634    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:13:52.491549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.491522    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:52.563691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r56m\" (UniqueName: \"kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.565649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563706    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.565649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563732    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.565649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.565649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.565649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.563917    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.664921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.664895    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r56m\" (UniqueName: \"kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665015 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.664935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665015 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.664963    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665092 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665035    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665092 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665069    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665162 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665439    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.665450 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.665430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.666949 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.666932    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.667281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.667266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.671905 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.671882    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r56m\" (UniqueName: \"kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m\") pod \"rate-test-model-650a99-kserve-595df8bb7-6zv6t\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") " pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:52.779043 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:52.778966    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:13:53.052841 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.052814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:13:53.054225 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:13:53.054194    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ab320a8_1097_45e8_8070_a5009d4eb7e6.slice/crio-82d3e510175adffbff47dc9fedf68e08f331ef655cdb05728b5269c4a1bd7d9e WatchSource:0}: Error finding container 82d3e510175adffbff47dc9fedf68e08f331ef655cdb05728b5269c4a1bd7d9e: Status 404 returned error can't find the container with id 82d3e510175adffbff47dc9fedf68e08f331ef655cdb05728b5269c4a1bd7d9e
Sep 09 13:13:53.182803 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.182770    2580 generic.go:356] "Generic (PLEG): container finished" podID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerID="084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b" exitCode=0
Sep 09 13:13:53.182961 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.182846    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerDied","Data":"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"}
Sep 09 13:13:53.183318 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.183301    2580 scope.go:117] "RemoveContainer" containerID="084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"
Sep 09 13:13:53.184592 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.184560    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:53.184695 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.184651    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:53.184818 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.184704    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 09 13:13:53.184818 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.184730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" event={"ID":"9ab320a8-1097-45e8-8070-a5009d4eb7e6","Type":"ContainerStarted","Data":"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"}
Sep 09 13:13:53.184818 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.184760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" event={"ID":"9ab320a8-1097-45e8-8070-a5009d4eb7e6","Type":"ContainerStarted","Data":"82d3e510175adffbff47dc9fedf68e08f331ef655cdb05728b5269c4a1bd7d9e"}
Sep 09 13:13:53.218771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:53.218717    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" podStartSLOduration=1.218694834 podStartE2EDuration="1.218694834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:13:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:13:53.217358067 +0000 UTC m=+1531.252605975" watchObservedRunningTime="2026-09-09 13:13:53.218694834 +0000 UTC m=+1531.253942741"
Sep 09 13:13:54.192662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:54.192622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerStarted","Data":"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"}
Sep 09 13:13:54.750689 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:54.750648    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:54.750933 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:54.750909    2580 patch_prober.go:28] interesting pod/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 13:13:54.750990 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:54.750970    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 13:13:55.750224 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:55.750193    2580 patch_prober.go:28] interesting pod/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 13:13:55.750623 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:55.750254    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 13:13:56.750297 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:56.750256    2580 patch_prober.go:28] interesting pod/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 13:13:56.750812 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:56.750312    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 13:13:57.114533 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:57.114421    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:57.750910 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:57.750872    2580 patch_prober.go:28] interesting pod/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 13:13:57.751322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:57.750936    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 13:13:58.751000 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:58.750960    2580 patch_prober.go:28] interesting pod/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused" start-of-body=
Sep 09 13:13:58.751390 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:58.751018    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.88:15021/healthz/ready\": dial tcp 10.132.0.88:15021: connect: connection refused"
Sep 09 13:13:59.134484 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:59.134388    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:13:59.755009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:59.754977    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:59.755392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:59.755232    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:13:59.755985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:13:59.755963    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:14:02.307618 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.307584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"]
Sep 09 13:14:02.311746 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.311722    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:02.313893 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.313865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:14:02.319352 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.319326    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"]
Sep 09 13:14:02.355795 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.355759    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-21da34-gkqrt\" (UID: \"fafb3a6a-ceba-42bf-af04-bb4f39eb4941\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:02.425277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.425247    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:02.457299 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.457267    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-21da34-gkqrt\" (UID: \"fafb3a6a-ceba-42bf-af04-bb4f39eb4941\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:02.457906 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.457883    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-21da34-gkqrt\" (UID: \"fafb3a6a-ceba-42bf-af04-bb4f39eb4941\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:02.626861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.626778    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:02.779583 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.779551    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:14:02.779762 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.779625    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:14:02.792530 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.792491    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:14:02.896691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:02.896668    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"]
Sep 09 13:14:02.897801 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:14:02.897779    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafb3a6a_ceba_42bf_af04_bb4f39eb4941.slice/crio-7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7 WatchSource:0}: Error finding container 7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7: Status 404 returned error can't find the container with id 7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7
Sep 09 13:14:03.257131 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:03.257093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt" event={"ID":"fafb3a6a-ceba-42bf-af04-bb4f39eb4941","Type":"ContainerStarted","Data":"582f7a1a2f15779491e92b20884e260278b0bd3342b8a16f75ea1221246ce63d"}
Sep 09 13:14:03.257260 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:03.257155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt" event={"ID":"fafb3a6a-ceba-42bf-af04-bb4f39eb4941","Type":"ContainerStarted","Data":"7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7"}
Sep 09 13:14:03.268990 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:03.268964    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:14:03.273256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:03.273216    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt" podStartSLOduration=1.273202087 podStartE2EDuration="1.273202087s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:03.270710241 +0000 UTC m=+1541.305958149" watchObservedRunningTime="2026-09-09 13:14:03.273202087 +0000 UTC m=+1541.308450030"
Sep 09 13:14:04.265414 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:04.265381    2580 generic.go:356] "Generic (PLEG): container finished" podID="fafb3a6a-ceba-42bf-af04-bb4f39eb4941" containerID="582f7a1a2f15779491e92b20884e260278b0bd3342b8a16f75ea1221246ce63d" exitCode=0
Sep 09 13:14:04.265833 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:04.265467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt" event={"ID":"fafb3a6a-ceba-42bf-af04-bb4f39eb4941","Type":"ContainerDied","Data":"582f7a1a2f15779491e92b20884e260278b0bd3342b8a16f75ea1221246ce63d"}
Sep 09 13:14:04.663820 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:04.663778    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:05.413405 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:05.413378    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:05.483270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:05.483239    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca\") pod \"fafb3a6a-ceba-42bf-af04-bb4f39eb4941\" (UID: \"fafb3a6a-ceba-42bf-af04-bb4f39eb4941\") "
Sep 09 13:14:05.483605 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:05.483584    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fafb3a6a-ceba-42bf-af04-bb4f39eb4941" (UID: "fafb3a6a-ceba-42bf-af04-bb4f39eb4941"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:05.584119 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:05.584046    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fafb3a6a-ceba-42bf-af04-bb4f39eb4941-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:06.279074 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:06.279048    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"
Sep 09 13:14:06.279250 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:06.279127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt" event={"ID":"fafb3a6a-ceba-42bf-af04-bb4f39eb4941","Type":"ContainerDied","Data":"7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7"}
Sep 09 13:14:06.279250 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:06.279167    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dadc1f92bb520882efa2169afa35125433b6dc16a94acc9b904dc6e9134c3e7"
Sep 09 13:14:12.435753 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:12.435713    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:13.277713 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:13.277679    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.007774 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.007736    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.085006 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.084966    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.148252 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.148220    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.204209 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.204175    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"]
Sep 09 13:14:18.206654 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.206621    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-21da34-gkqrt"]
Sep 09 13:14:18.211725 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.211685    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.281719 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.281635    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:18.454336 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.454294    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fafb3a6a-ceba-42bf-af04-bb4f39eb4941" path="/var/lib/kubelet/pods/fafb3a6a-ceba-42bf-af04-bb4f39eb4941/volumes"
Sep 09 13:14:18.588181 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:18.588092    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:24.351838 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.351790    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:24.418096 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.418053    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:24.624050 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.623971    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:24.973679 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.973647    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"]
Sep 09 13:14:24.974129 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.974113    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fafb3a6a-ceba-42bf-af04-bb4f39eb4941" containerName="revoke-keys"
Sep 09 13:14:24.974129 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.974130    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="fafb3a6a-ceba-42bf-af04-bb4f39eb4941" containerName="revoke-keys"
Sep 09 13:14:24.974232 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.974199    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="fafb3a6a-ceba-42bf-af04-bb4f39eb4941" containerName="revoke-keys"
Sep 09 13:14:24.977425 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.977409    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:24.981254 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.981234    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:14:24.990599 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:24.990566    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"]
Sep 09 13:14:25.061415 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.061384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz\" (UID: \"a42f9ffb-bc9d-4bb9-acad-41275edc5da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:25.106308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.106276    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:25.162658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.162620    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz\" (UID: \"a42f9ffb-bc9d-4bb9-acad-41275edc5da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:25.163261 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.163239    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz\" (UID: \"a42f9ffb-bc9d-4bb9-acad-41275edc5da4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:25.288238 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.288157    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:25.556949 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:25.556919    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"]
Sep 09 13:14:25.558227 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:14:25.558196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda42f9ffb_bc9d_4bb9_acad_41275edc5da4.slice/crio-cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88 WatchSource:0}: Error finding container cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88: Status 404 returned error can't find the container with id cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88
Sep 09 13:14:26.412176 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:26.412138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz" event={"ID":"a42f9ffb-bc9d-4bb9-acad-41275edc5da4","Type":"ContainerStarted","Data":"1dab634c0fbabd174fd6e94a5845831f456355285bae1246dc6d303624b4a3df"}
Sep 09 13:14:26.412318 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:26.412218    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz" event={"ID":"a42f9ffb-bc9d-4bb9-acad-41275edc5da4","Type":"ContainerStarted","Data":"cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88"}
Sep 09 13:14:26.428029 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:26.427973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz" podStartSLOduration=2.427954437 podStartE2EDuration="2.427954437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:26.42777586 +0000 UTC m=+1564.463023770" watchObservedRunningTime="2026-09-09 13:14:26.427954437 +0000 UTC m=+1564.463202367"
Sep 09 13:14:27.420386 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:27.420349    2580 generic.go:356] "Generic (PLEG): container finished" podID="a42f9ffb-bc9d-4bb9-acad-41275edc5da4" containerID="1dab634c0fbabd174fd6e94a5845831f456355285bae1246dc6d303624b4a3df" exitCode=0
Sep 09 13:14:27.420386 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:27.420387    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz" event={"ID":"a42f9ffb-bc9d-4bb9-acad-41275edc5da4","Type":"ContainerDied","Data":"1dab634c0fbabd174fd6e94a5845831f456355285bae1246dc6d303624b4a3df"}
Sep 09 13:14:28.567861 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:28.567836    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:28.694077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:28.693993    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca\") pod \"a42f9ffb-bc9d-4bb9-acad-41275edc5da4\" (UID: \"a42f9ffb-bc9d-4bb9-acad-41275edc5da4\") "
Sep 09 13:14:28.694373 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:28.694348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a42f9ffb-bc9d-4bb9-acad-41275edc5da4" (UID: "a42f9ffb-bc9d-4bb9-acad-41275edc5da4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:28.795512 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:28.795483    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a42f9ffb-bc9d-4bb9-acad-41275edc5da4-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:29.435347 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:29.435316    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"
Sep 09 13:14:29.435577 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:29.435350    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz" event={"ID":"a42f9ffb-bc9d-4bb9-acad-41275edc5da4","Type":"ContainerDied","Data":"cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88"}
Sep 09 13:14:29.435577 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:29.435396    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd003d3df9e6a2acac52010317b912b2a44ae7f0924de1926d5ddde781d05d88"
Sep 09 13:14:40.431169 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.431125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:40.458311 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.458279    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:14:40.458571 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.458520    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" podUID="813eb172-0739-4a92-b0ce-2cbd34357cb5" containerName="payload-processing" containerID="cri-o://e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6" gracePeriod=30
Sep 09 13:14:40.527844 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.527810    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:40.609330 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.609298    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"]
Sep 09 13:14:40.611707 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.611423    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:40.613700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.613674    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-a813cb-mjgfz"]
Sep 09 13:14:40.666722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.666690    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:40.715613 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.715590    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:14:40.809822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.809792    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume\") pod \"813eb172-0739-4a92-b0ce-2cbd34357cb5\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") "
Sep 09 13:14:40.810005 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.809925    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nj5nq\" (UniqueName: \"kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq\") pod \"813eb172-0739-4a92-b0ce-2cbd34357cb5\" (UID: \"813eb172-0739-4a92-b0ce-2cbd34357cb5\") "
Sep 09 13:14:40.810168 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.810143    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "813eb172-0739-4a92-b0ce-2cbd34357cb5" (UID: "813eb172-0739-4a92-b0ce-2cbd34357cb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:40.812423 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.812393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq" (OuterVolumeSpecName: "kube-api-access-nj5nq") pod "813eb172-0739-4a92-b0ce-2cbd34357cb5" (UID: "813eb172-0739-4a92-b0ce-2cbd34357cb5"). InnerVolumeSpecName "kube-api-access-nj5nq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:40.910979 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.910944    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nj5nq\" (UniqueName: \"kubernetes.io/projected/813eb172-0739-4a92-b0ce-2cbd34357cb5-kube-api-access-nj5nq\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:40.911101 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.910981    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/813eb172-0739-4a92-b0ce-2cbd34357cb5-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:40.994833 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:40.994733    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:41.517837 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.517806    2580 generic.go:356] "Generic (PLEG): container finished" podID="813eb172-0739-4a92-b0ce-2cbd34357cb5" containerID="e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6" exitCode=0
Sep 09 13:14:41.518308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.517847    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"
Sep 09 13:14:41.518308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.517892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" event={"ID":"813eb172-0739-4a92-b0ce-2cbd34357cb5","Type":"ContainerDied","Data":"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"}
Sep 09 13:14:41.518308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.517942    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h" event={"ID":"813eb172-0739-4a92-b0ce-2cbd34357cb5","Type":"ContainerDied","Data":"732db7f8ec555c5f35f3853e452997cec61d5b4e90265d8a21dd6549e2d7daa0"}
Sep 09 13:14:41.518308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.517966    2580 scope.go:117] "RemoveContainer" containerID="e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"
Sep 09 13:14:41.527748 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.527731    2580 scope.go:117] "RemoveContainer" containerID="e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"
Sep 09 13:14:41.528039 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:14:41.528019    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6\": container with ID starting with e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6 not found: ID does not exist" containerID="e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"
Sep 09 13:14:41.528083 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.528047    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6"} err="failed to get container status \"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6\": rpc error: code = NotFound desc = could not find container \"e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6\": container with ID starting with e3424164f120585d44ea4b75c7331b0d7accc52efaa160c8895b9dd4f2f5a4b6 not found: ID does not exist"
Sep 09 13:14:41.539616 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.539594    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:14:41.545020 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:41.544995    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-a813cb-cf9dc5fd5-l8j8h"]
Sep 09 13:14:42.453698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:42.453664    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="813eb172-0739-4a92-b0ce-2cbd34357cb5" path="/var/lib/kubelet/pods/813eb172-0739-4a92-b0ce-2cbd34357cb5/volumes"
Sep 09 13:14:42.454054 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:42.454040    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a42f9ffb-bc9d-4bb9-acad-41275edc5da4" path="/var/lib/kubelet/pods/a42f9ffb-bc9d-4bb9-acad-41275edc5da4/volumes"
Sep 09 13:14:45.813490 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:45.813451    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:46.794689 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:46.794658    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:46.892659 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:46.892623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:47.090210 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.090117    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:14:47.090404 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.090375    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy" containerID="cri-o://17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d" gracePeriod=30
Sep 09 13:14:47.115306 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.115275    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:47.456300 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456268    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"]
Sep 09 13:14:47.456731 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456717    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="813eb172-0739-4a92-b0ce-2cbd34357cb5" containerName="payload-processing"
Sep 09 13:14:47.456782 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456732    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="813eb172-0739-4a92-b0ce-2cbd34357cb5" containerName="payload-processing"
Sep 09 13:14:47.456782 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456744    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a42f9ffb-bc9d-4bb9-acad-41275edc5da4" containerName="revoke-keys"
Sep 09 13:14:47.456782 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456749    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a42f9ffb-bc9d-4bb9-acad-41275edc5da4" containerName="revoke-keys"
Sep 09 13:14:47.456883 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456813    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="813eb172-0739-4a92-b0ce-2cbd34357cb5" containerName="payload-processing"
Sep 09 13:14:47.456883 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.456822    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a42f9ffb-bc9d-4bb9-acad-41275edc5da4" containerName="revoke-keys"
Sep 09 13:14:47.461349 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.461333    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:47.463796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.463767    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:14:47.467420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.467398    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"]
Sep 09 13:14:47.574652 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.574622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77\" (UID: \"032185fd-002e-44ae-ab33-9985ffe4b836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:47.589031 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.589001    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:47.675376 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.675348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77\" (UID: \"032185fd-002e-44ae-ab33-9985ffe4b836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:47.676007 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.675988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77\" (UID: \"032185fd-002e-44ae-ab33-9985ffe4b836\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:47.775994 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:47.775922    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:48.047624 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.047593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"]
Sep 09 13:14:48.049570 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:14:48.049539    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod032185fd_002e_44ae_ab33_9985ffe4b836.slice/crio-6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee WatchSource:0}: Error finding container 6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee: Status 404 returned error can't find the container with id 6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee
Sep 09 13:14:48.563616 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.563570    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77" event={"ID":"032185fd-002e-44ae-ab33-9985ffe4b836","Type":"ContainerStarted","Data":"917fe9fde3e4957feeafc5d4f31d542d3b89de44a080fbcf7f186ee8091004ae"}
Sep 09 13:14:48.563795 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.563638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77" event={"ID":"032185fd-002e-44ae-ab33-9985ffe4b836","Type":"ContainerStarted","Data":"6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee"}
Sep 09 13:14:48.576974 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.576879    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77" podStartSLOduration=1.576863823 podStartE2EDuration="1.576863823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:48.576645693 +0000 UTC m=+1586.611893595" watchObservedRunningTime="2026-09-09 13:14:48.576863823 +0000 UTC m=+1586.612111729"
Sep 09 13:14:48.882367 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.882328    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:14:48.898674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:48.898642    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:49.571522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:49.571468    2580 generic.go:356] "Generic (PLEG): container finished" podID="032185fd-002e-44ae-ab33-9985ffe4b836" containerID="917fe9fde3e4957feeafc5d4f31d542d3b89de44a080fbcf7f186ee8091004ae" exitCode=0
Sep 09 13:14:49.571932 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:49.571549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77" event={"ID":"032185fd-002e-44ae-ab33-9985ffe4b836","Type":"ContainerDied","Data":"917fe9fde3e4957feeafc5d4f31d542d3b89de44a080fbcf7f186ee8091004ae"}
Sep 09 13:14:49.945140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:49.944331    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:14:49.971893 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:49.971861    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:50.429418 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:50.429384    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:50.725828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:50.725793    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:50.803409 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:50.803383    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca\") pod \"032185fd-002e-44ae-ab33-9985ffe4b836\" (UID: \"032185fd-002e-44ae-ab33-9985ffe4b836\") "
Sep 09 13:14:50.803799 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:50.803778    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "032185fd-002e-44ae-ab33-9985ffe4b836" (UID: "032185fd-002e-44ae-ab33-9985ffe4b836"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:50.904297 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:50.904267    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/032185fd-002e-44ae-ab33-9985ffe4b836-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:51.128029 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:51.127934    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:51.228276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:51.228239    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:51.585596 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:51.585568    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"
Sep 09 13:14:51.585760 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:51.585600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77" event={"ID":"032185fd-002e-44ae-ab33-9985ffe4b836","Type":"ContainerDied","Data":"6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee"}
Sep 09 13:14:51.585760 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:51.585643    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6f195700b5592b4a40285ad458e02c1e533b25225d947202fa4acb7c82c17bee"
Sep 09 13:14:52.345441 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.345418    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:14:52.418161 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418132    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418192    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418302    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418357 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418341    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqcc4\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418388    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418487    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data\") pod \"dac442ec-30e4-4573-8f87-b2504c0e3435\" (UID: \"dac442ec-30e4-4573-8f87-b2504c0e3435\") "
Sep 09 13:14:52.418992 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.418964    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:52.419287 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.419114    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:52.419287 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.419255    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:52.419602 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.419575    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:14:52.419750 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.419670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data" (OuterVolumeSpecName: "istio-data") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:52.421877 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.421853    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4" (OuterVolumeSpecName: "kube-api-access-qqcc4") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "kube-api-access-qqcc4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:52.422373 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.422342    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token" (OuterVolumeSpecName: "istio-token") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:14:52.422530 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.422485    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:14:52.422923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.422904    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "dac442ec-30e4-4573-8f87-b2504c0e3435" (UID: "dac442ec-30e4-4573-8f87-b2504c0e3435"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 13:14:52.519777 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519752    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-data\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519777 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519775    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-socket\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519784    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-podinfo\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519792    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-envoy\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519799    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/dac442ec-30e4-4573-8f87-b2504c0e3435-istiod-ca-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519807    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqcc4\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-kube-api-access-qqcc4\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519818    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-workload-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519826    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/dac442ec-30e4-4573-8f87-b2504c0e3435-credential-socket\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.519923 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.519834    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/dac442ec-30e4-4573-8f87-b2504c0e3435-istio-token\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:14:52.592970 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.592939    2580 generic.go:356] "Generic (PLEG): container finished" podID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerID="17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d" exitCode=0
Sep 09 13:14:52.593153 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.592989    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"
Sep 09 13:14:52.593153 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.593006    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerDied","Data":"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"}
Sep 09 13:14:52.593153 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.593054    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz" event={"ID":"dac442ec-30e4-4573-8f87-b2504c0e3435","Type":"ContainerDied","Data":"a350390a0ae4c75c7f81853c986fd775dd9bc7afe7243d8abebb92ebd2d8d86f"}
Sep 09 13:14:52.593153 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.593076    2580 scope.go:117] "RemoveContainer" containerID="17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"
Sep 09 13:14:52.605281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.605264    2580 scope.go:117] "RemoveContainer" containerID="084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"
Sep 09 13:14:52.611154 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.611131    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:14:52.615100 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.615076    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-a813cb-openshift-default-7c88645694-xcqjz"]
Sep 09 13:14:52.615358 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.615345    2580 scope.go:117] "RemoveContainer" containerID="17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"
Sep 09 13:14:52.615683 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:14:52.615661    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d\": container with ID starting with 17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d not found: ID does not exist" containerID="17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"
Sep 09 13:14:52.615787 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.615689    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d"} err="failed to get container status \"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d\": rpc error: code = NotFound desc = could not find container \"17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d\": container with ID starting with 17666b781eaa6270f6c22053bd4e4a93c53c8514cb8805e6708c380a49f36c2d not found: ID does not exist"
Sep 09 13:14:52.615787 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.615705    2580 scope.go:117] "RemoveContainer" containerID="084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"
Sep 09 13:14:52.615977 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:14:52.615959    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b\": container with ID starting with 084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b not found: ID does not exist" containerID="084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"
Sep 09 13:14:52.616039 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:52.615983    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b"} err="failed to get container status \"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b\": rpc error: code = NotFound desc = could not find container \"084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b\": container with ID starting with 084b35442934197419dadfa32e17424511bce6199ae762d33da381fcca7d1e0b not found: ID does not exist"
Sep 09 13:14:53.872794 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:53.872734    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:53.960559 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:53.960495    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:54.453665 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:54.453637    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" path="/var/lib/kubelet/pods/dac442ec-30e4-4573-8f87-b2504c0e3435/volumes"
Sep 09 13:14:55.208942 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.208897    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:55.275151 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.275120    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:55.375108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.375064    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:55.664295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.664263    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:55.754682 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.754653    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:14:55.755068 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755056    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:14:55.755122 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755071    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:14:55.755122 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755095    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="032185fd-002e-44ae-ab33-9985ffe4b836" containerName="revoke-keys"
Sep 09 13:14:55.755122 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755101    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="032185fd-002e-44ae-ab33-9985ffe4b836" containerName="revoke-keys"
Sep 09 13:14:55.755259 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755173    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="032185fd-002e-44ae-ab33-9985ffe4b836" containerName="revoke-keys"
Sep 09 13:14:55.755259 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755195    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:14:55.755259 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.755204    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:14:55.756478 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.756450    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:55.758662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.758646    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.760797 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.760774    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-458b0f\""
Sep 09 13:14:55.769685 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.769662    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:14:55.848588 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.848555    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w8fs\" (UniqueName: \"kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.848732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.848634    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.949522 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.949424    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w8fs\" (UniqueName: \"kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.949687 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.949522    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.950197 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.950177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:55.956543 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:55.956495    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w8fs\" (UniqueName: \"kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs\") pod \"payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:56.071030 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.070996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:56.343165 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.343135    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:14:56.344354 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:14:56.344324    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a913dde_ba74_4b64_9d02_cdd7a9fa924e.slice/crio-f26ee84464fcce044ef3c4fe06db7c91e7ed43bf9009e23471a234a6478f9021 WatchSource:0}: Error finding container f26ee84464fcce044ef3c4fe06db7c91e7ed43bf9009e23471a234a6478f9021: Status 404 returned error can't find the container with id f26ee84464fcce044ef3c4fe06db7c91e7ed43bf9009e23471a234a6478f9021
Sep 09 13:14:56.624425 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.624333    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" event={"ID":"1a913dde-ba74-4b64-9d02-cdd7a9fa924e","Type":"ContainerStarted","Data":"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"}
Sep 09 13:14:56.624425 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.624379    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" event={"ID":"1a913dde-ba74-4b64-9d02-cdd7a9fa924e","Type":"ContainerStarted","Data":"f26ee84464fcce044ef3c4fe06db7c91e7ed43bf9009e23471a234a6478f9021"}
Sep 09 13:14:56.624706 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.624427    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:14:56.640740 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:56.640696    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" podStartSLOduration=1.640683401 podStartE2EDuration="1.640683401s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:14:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:14:56.638572693 +0000 UTC m=+1594.673820600" watchObservedRunningTime="2026-09-09 13:14:56.640683401 +0000 UTC m=+1594.675931307"
Sep 09 13:14:57.577858 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:57.577818    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.109486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.109445    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.179728 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.179695    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.253079 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.253038    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.353124 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.353089    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.370228 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.370118    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"]
Sep 09 13:14:58.372902 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.372880    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-0b0a1a-ndj77"]
Sep 09 13:14:58.452572 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.452536    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:14:58.455835 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.455809    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="032185fd-002e-44ae-ab33-9985ffe4b836" path="/var/lib/kubelet/pods/032185fd-002e-44ae-ab33-9985ffe4b836/volumes"
Sep 09 13:14:58.753781 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:14:58.753744    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:00.131771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.131732    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"]
Sep 09 13:15:00.132470 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.132420    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:15:00.132470 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.132441    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="dac442ec-30e4-4573-8f87-b2504c0e3435" containerName="istio-proxy"
Sep 09 13:15:00.137127 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.137105    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"]
Sep 09 13:15:00.137246 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.137232    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"
Sep 09 13:15:00.139579 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.139560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:15:00.140600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.140584    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"]
Sep 09 13:15:00.140715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.140694    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"
Sep 09 13:15:00.144726 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.144253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"]
Sep 09 13:15:00.144726 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.144345    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"
Sep 09 13:15:00.147637 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.147611    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"]
Sep 09 13:15:00.153533 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.153486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"
Sep 09 13:15:00.162104 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.161999    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"]
Sep 09 13:15:00.163315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.163272    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"
Sep 09 13:15:00.202896 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.202869    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"
Sep 09 13:15:00.435154 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.435130    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"]
Sep 09 13:15:00.436233 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:15:00.436207    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d0466ab_1616_4a2b_bc75_c195f44b92fa.slice/crio-b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c WatchSource:0}: Error finding container b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c: Status 404 returned error can't find the container with id b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c
Sep 09 13:15:00.456287 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.456268    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"]
Sep 09 13:15:00.457378 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:15:00.457355    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb00ef94_84e6_48f9_a396_b73fd9b01f09.slice/crio-61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c WatchSource:0}: Error finding container 61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c: Status 404 returned error can't find the container with id 61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c
Sep 09 13:15:00.506072 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.506038    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"]
Sep 09 13:15:00.506949 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:15:00.506930    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f556c28_ef1d_4105_9153_c8fd0ca87677.slice/crio-bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d WatchSource:0}: Error finding container bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d: Status 404 returned error can't find the container with id bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d
Sep 09 13:15:00.649984 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.649950    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64" event={"ID":"7f556c28-ef1d-4105-9153-c8fd0ca87677","Type":"ContainerStarted","Data":"bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d"}
Sep 09 13:15:00.651191 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.651169    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875" event={"ID":"7d0466ab-1616-4a2b-bc75-c195f44b92fa","Type":"ContainerStarted","Data":"b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c"}
Sep 09 13:15:00.652342 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:00.652322    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4" event={"ID":"eb00ef94-84e6-48f9-a396-b73fd9b01f09","Type":"ContainerStarted","Data":"61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c"}
Sep 09 13:15:01.659884 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.659845    2580 generic.go:356] "Generic (PLEG): container finished" podID="7f556c28-ef1d-4105-9153-c8fd0ca87677" containerID="4e28a2668a3d1c6615b00243e522f2e3fff0d546be3c2d3255897fb00cb9f929" exitCode=0
Sep 09 13:15:01.660281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.659933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64" event={"ID":"7f556c28-ef1d-4105-9153-c8fd0ca87677","Type":"ContainerDied","Data":"4e28a2668a3d1c6615b00243e522f2e3fff0d546be3c2d3255897fb00cb9f929"}
Sep 09 13:15:01.661546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.661525    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d0466ab-1616-4a2b-bc75-c195f44b92fa" containerID="04847cb13c11946b16b1e5993cc932b482740f0d8026a91880c8c0ff0b5fe1e7" exitCode=0
Sep 09 13:15:01.661658 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.661580    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875" event={"ID":"7d0466ab-1616-4a2b-bc75-c195f44b92fa","Type":"ContainerDied","Data":"04847cb13c11946b16b1e5993cc932b482740f0d8026a91880c8c0ff0b5fe1e7"}
Sep 09 13:15:01.663304 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.663280    2580 generic.go:356] "Generic (PLEG): container finished" podID="eb00ef94-84e6-48f9-a396-b73fd9b01f09" containerID="8ed252c580c6e4fb9bd77ab283b2e44a67ba444b045616451c63fdb865e321d7" exitCode=0
Sep 09 13:15:01.663400 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:01.663356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4" event={"ID":"eb00ef94-84e6-48f9-a396-b73fd9b01f09","Type":"ContainerDied","Data":"8ed252c580c6e4fb9bd77ab283b2e44a67ba444b045616451c63fdb865e321d7"}
Sep 09 13:15:02.867008 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:02.866985    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"
Sep 09 13:15:02.924126 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:02.924055    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"
Sep 09 13:15:02.927726 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:02.927704    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"
Sep 09 13:15:03.679197 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.679169    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"
Sep 09 13:15:03.679381 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.679166    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875" event={"ID":"7d0466ab-1616-4a2b-bc75-c195f44b92fa","Type":"ContainerDied","Data":"b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c"}
Sep 09 13:15:03.679381 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.679316    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b891dcd7c230cf28544a090a0c59f2cf633c593e13f6b2ef3d8cdeadd2e1ca7c"
Sep 09 13:15:03.680674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.680649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"
Sep 09 13:15:03.680810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.680688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4" event={"ID":"eb00ef94-84e6-48f9-a396-b73fd9b01f09","Type":"ContainerDied","Data":"61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c"}
Sep 09 13:15:03.680810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.680772    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="61de750a98c24be251294e1243bc95e0ae694df5e53afc02ef2822f57e3b813c"
Sep 09 13:15:03.682249 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.682230    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"
Sep 09 13:15:03.682367 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.682245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64" event={"ID":"7f556c28-ef1d-4105-9153-c8fd0ca87677","Type":"ContainerDied","Data":"bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d"}
Sep 09 13:15:03.682367 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.682288    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbf1d2c6d45f094fccd75f1e8e6524513a8b04773b37734a9a4f22dc1ac57b8d"
Sep 09 13:15:03.904421 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:03.904368    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" podUID="7e623355-2b1f-45a9-89a0-fa934408f0a7" containerName="main" containerID="cri-o://e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e" gracePeriod=45
Sep 09 13:15:04.158564 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.158541    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:15:04.229630 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4sms\" (UniqueName: \"kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.229790 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229696    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.229790 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229730    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.229790 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229767    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.229969 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.229969 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229883    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir\") pod \"7e623355-2b1f-45a9-89a0-fa934408f0a7\" (UID: \"7e623355-2b1f-45a9-89a0-fa934408f0a7\") "
Sep 09 13:15:04.230086 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.229996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home" (OuterVolumeSpecName: "home") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:04.230086 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.230074    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache" (OuterVolumeSpecName: "model-cache") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:04.230209 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.230164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:04.230288 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.230271    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.230353 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.230290    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.230353 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.230317    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.231854 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.231833    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms" (OuterVolumeSpecName: "kube-api-access-n4sms") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "kube-api-access-n4sms". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:04.231949 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.231869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm" (OuterVolumeSpecName: "dshm") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:04.231949 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.231881    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "7e623355-2b1f-45a9-89a0-fa934408f0a7" (UID: "7e623355-2b1f-45a9-89a0-fa934408f0a7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:15:04.331673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.331611    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4sms\" (UniqueName: \"kubernetes.io/projected/7e623355-2b1f-45a9-89a0-fa934408f0a7-kube-api-access-n4sms\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.331673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.331634    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7e623355-2b1f-45a9-89a0-fa934408f0a7-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.331673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.331646    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7e623355-2b1f-45a9-89a0-fa934408f0a7-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:04.686766 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.686716    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:04.690959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.690931    2580 generic.go:356] "Generic (PLEG): container finished" podID="7e623355-2b1f-45a9-89a0-fa934408f0a7" containerID="e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e" exitCode=0
Sep 09 13:15:04.691087 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.690977    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"
Sep 09 13:15:04.691087 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.691013    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" event={"ID":"7e623355-2b1f-45a9-89a0-fa934408f0a7","Type":"ContainerDied","Data":"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"}
Sep 09 13:15:04.691087 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.691061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n" event={"ID":"7e623355-2b1f-45a9-89a0-fa934408f0a7","Type":"ContainerDied","Data":"a660b6c36199f3686bcca1e8dba4662450a89242fee63803b10f7e93c59c7bb4"}
Sep 09 13:15:04.691087 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.691082    2580 scope.go:117] "RemoveContainer" containerID="e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"
Sep 09 13:15:04.701279 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.701259    2580 scope.go:117] "RemoveContainer" containerID="e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"
Sep 09 13:15:04.701550 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:15:04.701532    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e\": container with ID starting with e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e not found: ID does not exist" containerID="e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"
Sep 09 13:15:04.701613 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.701556    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e"} err="failed to get container status \"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e\": rpc error: code = NotFound desc = could not find container \"e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e\": container with ID starting with e3e5c8d4ed09dda51fe153dae83f4a97f3c12b53ef6133126c967815ed4d6b2e not found: ID does not exist"
Sep 09 13:15:04.708910 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.708889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:15:04.716484 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.716463    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-ada1ae/rate-test-model-ada1ae-kserve-799f69769f-bnm2n"]
Sep 09 13:15:04.755325 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.755296    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:04.937673 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.937592    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:04.969587 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:04.969547    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" podUID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" containerName="main" containerID="cri-o://e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477" gracePeriod=45
Sep 09 13:15:05.222947 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.222918    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:15:05.277511 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.277465    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"]
Sep 09 13:15:05.278021 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278002    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f556c28-ef1d-4105-9153-c8fd0ca87677" containerName="cleanup"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278024    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f556c28-ef1d-4105-9153-c8fd0ca87677" containerName="cleanup"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278047    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb00ef94-84e6-48f9-a396-b73fd9b01f09" containerName="cleanup"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278056    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb00ef94-84e6-48f9-a396-b73fd9b01f09" containerName="cleanup"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278073    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" containerName="main"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278083    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" containerName="main"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278109    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e623355-2b1f-45a9-89a0-fa934408f0a7" containerName="main"
Sep 09 13:15:05.278120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278117    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e623355-2b1f-45a9-89a0-fa934408f0a7" containerName="main"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278137    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d0466ab-1616-4a2b-bc75-c195f44b92fa" containerName="cleanup"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278145    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d0466ab-1616-4a2b-bc75-c195f44b92fa" containerName="cleanup"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278249    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d0466ab-1616-4a2b-bc75-c195f44b92fa" containerName="cleanup"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278264    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e623355-2b1f-45a9-89a0-fa934408f0a7" containerName="main"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278274    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f556c28-ef1d-4105-9153-c8fd0ca87677" containerName="cleanup"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278287    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb00ef94-84e6-48f9-a396-b73fd9b01f09" containerName="cleanup"
Sep 09 13:15:05.278479 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.278297    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" containerName="main"
Sep 09 13:15:05.281567 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.281550    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:05.283751 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.283733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:15:05.290604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.290582    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"]
Sep 09 13:15:05.341412 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341381    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341419    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341457    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341494    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341604 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4r56m\" (UniqueName: \"kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341851 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341634    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir\") pod \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\" (UID: \"9ab320a8-1097-45e8-8070-a5009d4eb7e6\") "
Sep 09 13:15:05.341851 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341715    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache" (OuterVolumeSpecName: "model-cache") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:05.341851 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341768    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home" (OuterVolumeSpecName: "home") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:05.341980 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341960    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748\" (UID: \"f1b3fd6e-479a-4545-a387-d725a49d8816\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:05.342027 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.341973    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:05.342088 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.342049    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.342088 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.342071    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.342088 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.342085    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.343807 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.343776    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:15:05.343924 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.343834    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm" (OuterVolumeSpecName: "dshm") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:15:05.343924 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.343861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m" (OuterVolumeSpecName: "kube-api-access-4r56m") pod "9ab320a8-1097-45e8-8070-a5009d4eb7e6" (UID: "9ab320a8-1097-45e8-8070-a5009d4eb7e6"). InnerVolumeSpecName "kube-api-access-4r56m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:05.395189 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.395160    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:05.443078 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.443049    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748\" (UID: \"f1b3fd6e-479a-4545-a387-d725a49d8816\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:05.443192 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.443173    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9ab320a8-1097-45e8-8070-a5009d4eb7e6-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.443237 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.443191    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r56m\" (UniqueName: \"kubernetes.io/projected/9ab320a8-1097-45e8-8070-a5009d4eb7e6-kube-api-access-4r56m\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.443237 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.443205    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9ab320a8-1097-45e8-8070-a5009d4eb7e6-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:05.443712 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.443695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748\" (UID: \"f1b3fd6e-479a-4545-a387-d725a49d8816\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:05.596438 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.596364    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:05.701322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.701052    2580 generic.go:356] "Generic (PLEG): container finished" podID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" containerID="e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477" exitCode=0
Sep 09 13:15:05.701322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.701098    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" event={"ID":"9ab320a8-1097-45e8-8070-a5009d4eb7e6","Type":"ContainerDied","Data":"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"}
Sep 09 13:15:05.701322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.701131    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"
Sep 09 13:15:05.701322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.701149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t" event={"ID":"9ab320a8-1097-45e8-8070-a5009d4eb7e6","Type":"ContainerDied","Data":"82d3e510175adffbff47dc9fedf68e08f331ef655cdb05728b5269c4a1bd7d9e"}
Sep 09 13:15:05.701322 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.701172    2580 scope.go:117] "RemoveContainer" containerID="e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"
Sep 09 13:15:05.712236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.712128    2580 scope.go:117] "RemoveContainer" containerID="e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"
Sep 09 13:15:05.712466 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:15:05.712443    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477\": container with ID starting with e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477 not found: ID does not exist" containerID="e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"
Sep 09 13:15:05.712577 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.712473    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477"} err="failed to get container status \"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477\": rpc error: code = NotFound desc = could not find container \"e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477\": container with ID starting with e2add686014f4e5845e53df340d4dc636e2f5e9fa62d2615bfc32f9136e8e477 not found: ID does not exist"
Sep 09 13:15:05.724103 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.724078    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:15:05.727278 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.727252    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-650a99/rate-test-model-650a99-kserve-595df8bb7-6zv6t"]
Sep 09 13:15:05.868281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:05.868253    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"]
Sep 09 13:15:05.869496 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:15:05.869473    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1b3fd6e_479a_4545_a387_d725a49d8816.slice/crio-bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f WatchSource:0}: Error finding container bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f: Status 404 returned error can't find the container with id bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f
Sep 09 13:15:06.453443 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:06.453413    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e623355-2b1f-45a9-89a0-fa934408f0a7" path="/var/lib/kubelet/pods/7e623355-2b1f-45a9-89a0-fa934408f0a7/volumes"
Sep 09 13:15:06.453869 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:06.453828    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab320a8-1097-45e8-8070-a5009d4eb7e6" path="/var/lib/kubelet/pods/9ab320a8-1097-45e8-8070-a5009d4eb7e6/volumes"
Sep 09 13:15:06.710605 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:06.710531    2580 generic.go:356] "Generic (PLEG): container finished" podID="f1b3fd6e-479a-4545-a387-d725a49d8816" containerID="ae87b9a481a05adc613a060a1d0fe5d476d2b065fd1fba1be4d6b35c9b01c8e6" exitCode=0
Sep 09 13:15:06.710741 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:06.710598    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748" event={"ID":"f1b3fd6e-479a-4545-a387-d725a49d8816","Type":"ContainerDied","Data":"ae87b9a481a05adc613a060a1d0fe5d476d2b065fd1fba1be4d6b35c9b01c8e6"}
Sep 09 13:15:06.710741 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:06.710646    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748" event={"ID":"f1b3fd6e-479a-4545-a387-d725a49d8816","Type":"ContainerStarted","Data":"bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f"}
Sep 09 13:15:07.093600 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.093494    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:07.144780 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.144743    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:07.633017 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.632987    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:15:07.855933 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.855910    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:07.967411 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.967382    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca\") pod \"f1b3fd6e-479a-4545-a387-d725a49d8816\" (UID: \"f1b3fd6e-479a-4545-a387-d725a49d8816\") "
Sep 09 13:15:07.967769 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:07.967744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f1b3fd6e-479a-4545-a387-d725a49d8816" (UID: "f1b3fd6e-479a-4545-a387-d725a49d8816"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:15:08.068302 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:08.068264    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f1b3fd6e-479a-4545-a387-d725a49d8816-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:08.727732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:08.727649    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"
Sep 09 13:15:08.727732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:08.727681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748" event={"ID":"f1b3fd6e-479a-4545-a387-d725a49d8816","Type":"ContainerDied","Data":"bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f"}
Sep 09 13:15:08.727732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:08.727728    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bbc5464351309338d83143f585ab86427156137d0e07f4241b83bd717941838f"
Sep 09 13:15:15.707413 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.707306    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"]
Sep 09 13:15:15.711193 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.711166    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-941953-29815995-95z64"]
Sep 09 13:15:15.743237 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.743204    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:15:15.743532 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.743467    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" podUID="c529b246-dd89-4741-a7df-123e4c3c1098" containerName="payload-pre-processing" containerID="cri-o://8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39" gracePeriod=30
Sep 09 13:15:15.870112 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.870077    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:15.894887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.894854    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"]
Sep 09 13:15:15.897338 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.897311    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-941953-cz748"]
Sep 09 13:15:15.929933 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:15.929905    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:16.006108 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.006087    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:15:16.097540 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.097491    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:16.134554 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.134519    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume\") pod \"c529b246-dd89-4741-a7df-123e4c3c1098\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") "
Sep 09 13:15:16.134710 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.134580    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x7gbm\" (UniqueName: \"kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm\") pod \"c529b246-dd89-4741-a7df-123e4c3c1098\" (UID: \"c529b246-dd89-4741-a7df-123e4c3c1098\") "
Sep 09 13:15:16.134906 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.134886    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c529b246-dd89-4741-a7df-123e4c3c1098" (UID: "c529b246-dd89-4741-a7df-123e4c3c1098"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:15:16.136656 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.136636    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm" (OuterVolumeSpecName: "kube-api-access-x7gbm") pod "c529b246-dd89-4741-a7df-123e4c3c1098" (UID: "c529b246-dd89-4741-a7df-123e4c3c1098"). InnerVolumeSpecName "kube-api-access-x7gbm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:15:16.235496 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.235436    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c529b246-dd89-4741-a7df-123e4c3c1098-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.235496 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.235461    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x7gbm\" (UniqueName: \"kubernetes.io/projected/c529b246-dd89-4741-a7df-123e4c3c1098-kube-api-access-x7gbm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:15:16.295912 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.295873    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:16.453608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.453580    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f556c28-ef1d-4105-9153-c8fd0ca87677" path="/var/lib/kubelet/pods/7f556c28-ef1d-4105-9153-c8fd0ca87677/volumes"
Sep 09 13:15:16.453884 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.453871    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f1b3fd6e-479a-4545-a387-d725a49d8816" path="/var/lib/kubelet/pods/f1b3fd6e-479a-4545-a387-d725a49d8816/volumes"
Sep 09 13:15:16.776992 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.776960    2580 generic.go:356] "Generic (PLEG): container finished" podID="c529b246-dd89-4741-a7df-123e4c3c1098" containerID="8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39" exitCode=0
Sep 09 13:15:16.777420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.777005    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"
Sep 09 13:15:16.777420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.777041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" event={"ID":"c529b246-dd89-4741-a7df-123e4c3c1098","Type":"ContainerDied","Data":"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"}
Sep 09 13:15:16.777420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.777091    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4" event={"ID":"c529b246-dd89-4741-a7df-123e4c3c1098","Type":"ContainerDied","Data":"034238dee90a391ef67419b473b2c34b30510eef6dde063ff16d4aee4ca7cc83"}
Sep 09 13:15:16.777420 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.777111    2580 scope.go:117] "RemoveContainer" containerID="8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"
Sep 09 13:15:16.786463 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.786445    2580 scope.go:117] "RemoveContainer" containerID="8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"
Sep 09 13:15:16.786753 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:15:16.786737    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39\": container with ID starting with 8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39 not found: ID does not exist" containerID="8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"
Sep 09 13:15:16.786811 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.786761    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39"} err="failed to get container status \"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39\": rpc error: code = NotFound desc = could not find container \"8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39\": container with ID starting with 8aecd5ccd6ea995a96130404af229adf344b1b32b1f25902e25b8be7d38f3d39 not found: ID does not exist"
Sep 09 13:15:16.792575 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.792552    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:15:16.807219 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:16.807195    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-941953-55c9b46f78-krjk4"]
Sep 09 13:15:18.453623 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:18.453595    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c529b246-dd89-4741-a7df-123e4c3c1098" path="/var/lib/kubelet/pods/c529b246-dd89-4741-a7df-123e4c3c1098/volumes"
Sep 09 13:15:18.861920 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:18.861838    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:19.560652 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:19.560620    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:19.859269 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:19.859187    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:20.357081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:20.357046    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:22.138338 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:22.138300    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:22.237928 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:22.237896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:22.406407 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:22.406371    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:22.748691 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:22.748608    2580 scope.go:117] "RemoveContainer" containerID="0ed4a1840a10f8dcec76e192cc8cd726b7a96bccbaea5dcb6fc54723e478098a"
Sep 09 13:15:23.100204 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:23.100125    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:23.191269 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:23.191231    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.400307 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.400270    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.456860 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.456824    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.565835 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.565794    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.857283 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.857197    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.955888 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.955852    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:24.970133 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970102    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:15:24.970571 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970558    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c529b246-dd89-4741-a7df-123e4c3c1098" containerName="payload-pre-processing"
Sep 09 13:15:24.970625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970574    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c529b246-dd89-4741-a7df-123e4c3c1098" containerName="payload-pre-processing"
Sep 09 13:15:24.970625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970597    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f1b3fd6e-479a-4545-a387-d725a49d8816" containerName="revoke-keys"
Sep 09 13:15:24.970625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970605    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f1b3fd6e-479a-4545-a387-d725a49d8816" containerName="revoke-keys"
Sep 09 13:15:24.970723 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970671    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f1b3fd6e-479a-4545-a387-d725a49d8816" containerName="revoke-keys"
Sep 09 13:15:24.970723 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.970683    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c529b246-dd89-4741-a7df-123e4c3c1098" containerName="payload-pre-processing"
Sep 09 13:15:24.973722 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.973705    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:24.975992 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.975974    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-ae6936\""
Sep 09 13:15:24.976598 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.976574    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-ae6936-dockercfg-dscd6\""
Sep 09 13:15:24.985379 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:24.985359    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:15:25.114729 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.114642    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-447gq\" (UniqueName: \"kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.114894 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.114768    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.215813 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.215764    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.216011 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.215936    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-447gq\" (UniqueName: \"kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.216606 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.216577    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.224772 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.224742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-447gq\" (UniqueName: \"kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq\") pod \"payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.285193 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.285165    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.562845 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.562818    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:15:25.563671 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:15:25.563649    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86063bfb_9bcc_4302_bad9_1a9d513d5ea0.slice/crio-22e08946b4854f794df19db87422428e24792e0a8c081c281b27aaa08a2a79b2 WatchSource:0}: Error finding container 22e08946b4854f794df19db87422428e24792e0a8c081c281b27aaa08a2a79b2: Status 404 returned error can't find the container with id 22e08946b4854f794df19db87422428e24792e0a8c081c281b27aaa08a2a79b2
Sep 09 13:15:25.842898 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.842756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" event={"ID":"86063bfb-9bcc-4302-bad9-1a9d513d5ea0","Type":"ContainerStarted","Data":"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"}
Sep 09 13:15:25.842898 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.842839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" event={"ID":"86063bfb-9bcc-4302-bad9-1a9d513d5ea0","Type":"ContainerStarted","Data":"22e08946b4854f794df19db87422428e24792e0a8c081c281b27aaa08a2a79b2"}
Sep 09 13:15:25.842898 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.842867    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:25.858544 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:25.858480    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" podStartSLOduration=1.858465313 podStartE2EDuration="1.858465313s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:15:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:15:25.856713675 +0000 UTC m=+1623.891961583" watchObservedRunningTime="2026-09-09 13:15:25.858465313 +0000 UTC m=+1623.893713219"
Sep 09 13:15:36.851977 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:36.851945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:15:48.158716 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:48.158676    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:15:53.245417 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:15:53.245386    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:16:03.506204 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:03.506167    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:11.567993 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.567956    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:16:11.572333 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.572312    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.574775 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.574745    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-458b0f\"/\"kube-root-ca.crt\""
Sep 09 13:16:11.575944 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.575641    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-458b0f\"/\"openshift-service-ca.crt\""
Sep 09 13:16:11.576198 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.576174    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-458b0f\"/\"ipp-route-458b0f-kserve-self-signed-certs\""
Sep 09 13:16:11.576748 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.576724    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-458b0f\"/\"default-dockercfg-sngn5\""
Sep 09 13:16:11.580288 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.580264    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:16:11.588695 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.588670    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:11.731471 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.731674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731524    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.731674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731560    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.731674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731582    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5m2vv\" (UniqueName: \"kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.731674 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731654    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.731842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.731688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.832960    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833067    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833097    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5m2vv\" (UniqueName: \"kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833531 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833481    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833609 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.833663 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.833644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.835435 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.835407    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.835687 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.835670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.840694 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.840675    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5m2vv\" (UniqueName: \"kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv\") pod \"ipp-route-458b0f-kserve-7f64c8848c-n6d22\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") " pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:11.883310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:11.883282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:12.158275 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:12.158247    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:16:12.159144 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:16:12.159113    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4f03c2de_04e6_4edc_a460_0141cd54e458.slice/crio-6c69225d3faa7438bd503da1dc546b37b6d2cb38caf88f99d4c4f0281b851156 WatchSource:0}: Error finding container 6c69225d3faa7438bd503da1dc546b37b6d2cb38caf88f99d4c4f0281b851156: Status 404 returned error can't find the container with id 6c69225d3faa7438bd503da1dc546b37b6d2cb38caf88f99d4c4f0281b851156
Sep 09 13:16:13.147761 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:13.147718    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" event={"ID":"4f03c2de-04e6-4edc-a460-0141cd54e458","Type":"ContainerStarted","Data":"985b00fae6c78fd3ed97edae7dd951244f95cdb371120e711b315e8aa679adf5"}
Sep 09 13:16:13.147761 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:13.147760    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" event={"ID":"4f03c2de-04e6-4edc-a460-0141cd54e458","Type":"ContainerStarted","Data":"6c69225d3faa7438bd503da1dc546b37b6d2cb38caf88f99d4c4f0281b851156"}
Sep 09 13:16:13.164217 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:13.164172    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" podStartSLOduration=2.164156713 podStartE2EDuration="2.164156713s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:16:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:16:13.161444483 +0000 UTC m=+1671.196692389" watchObservedRunningTime="2026-09-09 13:16:13.164156713 +0000 UTC m=+1671.199404619"
Sep 09 13:16:21.884274 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:21.884234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:21.884747 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:21.884306    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:21.897164 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:21.897139    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:21.907662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:21.907639    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:22.217517 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:22.217482    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:16:22.797129 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:22.797096    2580 scope.go:117] "RemoveContainer" containerID="dcae99ee0b109be5f425eab9621611f1c84e4e634ef8fd77855110045a6f904a"
Sep 09 13:16:22.806019 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:22.805998    2580 scope.go:117] "RemoveContainer" containerID="14ed548262dab8cc455a056a4f0f9c2adc1fd0b44eedf5859c90caf3083da0c2"
Sep 09 13:16:24.109465 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:24.109426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:34.384982 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:34.384955    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:16:34.774120 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:34.774080    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:35.091386 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:35.091285    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:35.201305 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:35.201256    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:38.502388 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:38.502354    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:38.593843 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:38.593802    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:39.802459 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:39.802420    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:39.903356 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:39.903325    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:40.003529 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.003469    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:40.371788 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.371739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:16:40.375627 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.375611    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.378106 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.378085    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-964720\""
Sep 09 13:16:40.378246 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.378169    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-964720-dockercfg-46g2q\""
Sep 09 13:16:40.386834 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.386814    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:16:40.489756 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.489720    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.489905 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.489807    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzbnw\" (UniqueName: \"kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.591370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.591336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzbnw\" (UniqueName: \"kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.591539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.591436    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.592293 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.592269    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.598608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.598583    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzbnw\" (UniqueName: \"kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw\") pod \"payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.620751 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.620704    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:40.687887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.687855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:40.721295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.721260    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:16:40.962797 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:40.962769    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:16:40.965960 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:16:40.965932    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34776766_362c_4fcc_a188_431b7781732a.slice/crio-40c8180ef658a630f19c5c1f9a6cdf3b67f55e769dadaa934f0bad8b30311d45 WatchSource:0}: Error finding container 40c8180ef658a630f19c5c1f9a6cdf3b67f55e769dadaa934f0bad8b30311d45: Status 404 returned error can't find the container with id 40c8180ef658a630f19c5c1f9a6cdf3b67f55e769dadaa934f0bad8b30311d45
Sep 09 13:16:41.340732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:41.340649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" event={"ID":"34776766-362c-4fcc-a188-431b7781732a","Type":"ContainerStarted","Data":"3929464a18afb84cf1fd222e9b6691dbf3678075c62edbf1f995c4b73eca30d2"}
Sep 09 13:16:41.340732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:41.340703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" event={"ID":"34776766-362c-4fcc-a188-431b7781732a","Type":"ContainerStarted","Data":"40c8180ef658a630f19c5c1f9a6cdf3b67f55e769dadaa934f0bad8b30311d45"}
Sep 09 13:16:41.340932 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:41.340771    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:16:41.356206 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:41.356146    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" podStartSLOduration=1.3561268229999999 podStartE2EDuration="1.356126823s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:16:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:16:41.355144924 +0000 UTC m=+1699.390392832" watchObservedRunningTime="2026-09-09 13:16:41.356126823 +0000 UTC m=+1699.391374731"
Sep 09 13:16:52.348955 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:16:52.348923    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:17:03.548625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:03.548582    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:04.198889 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.198856    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"]
Sep 09 13:17:04.202820 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.202803    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:04.205889 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.205858    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:17:04.210192 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.210166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"]
Sep 09 13:17:04.212810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.211844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj\" (UID: \"3cea476b-153c-413f-a8dd-de062323dfe5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:04.312385 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.312351    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj\" (UID: \"3cea476b-153c-413f-a8dd-de062323dfe5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:04.312998 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.312973    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj\" (UID: \"3cea476b-153c-413f-a8dd-de062323dfe5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:04.517761 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.517685    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:04.792938 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:04.792910    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"]
Sep 09 13:17:04.794735 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:17:04.794708    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cea476b_153c_413f_a8dd_de062323dfe5.slice/crio-fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4 WatchSource:0}: Error finding container fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4: Status 404 returned error can't find the container with id fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4
Sep 09 13:17:05.491778 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:05.491734    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj" event={"ID":"3cea476b-153c-413f-a8dd-de062323dfe5","Type":"ContainerStarted","Data":"007e9e9fd477640d3c76707031e4fc2c0e2a8893199f0120c6aace8854ad2dda"}
Sep 09 13:17:05.491958 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:05.491804    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj" event={"ID":"3cea476b-153c-413f-a8dd-de062323dfe5","Type":"ContainerStarted","Data":"fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4"}
Sep 09 13:17:05.507897 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:05.507844    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj" podStartSLOduration=1.507829193 podStartE2EDuration="1.507829193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:17:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:17:05.505210653 +0000 UTC m=+1723.540458558" watchObservedRunningTime="2026-09-09 13:17:05.507829193 +0000 UTC m=+1723.543077100"
Sep 09 13:17:06.499467 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:06.499431    2580 generic.go:356] "Generic (PLEG): container finished" podID="3cea476b-153c-413f-a8dd-de062323dfe5" containerID="007e9e9fd477640d3c76707031e4fc2c0e2a8893199f0120c6aace8854ad2dda" exitCode=0
Sep 09 13:17:06.499879 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:06.499477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj" event={"ID":"3cea476b-153c-413f-a8dd-de062323dfe5","Type":"ContainerDied","Data":"007e9e9fd477640d3c76707031e4fc2c0e2a8893199f0120c6aace8854ad2dda"}
Sep 09 13:17:07.638937 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:07.638911    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:07.739736 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:07.739707    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca\") pod \"3cea476b-153c-413f-a8dd-de062323dfe5\" (UID: \"3cea476b-153c-413f-a8dd-de062323dfe5\") "
Sep 09 13:17:07.740090 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:07.740069    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3cea476b-153c-413f-a8dd-de062323dfe5" (UID: "3cea476b-153c-413f-a8dd-de062323dfe5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:07.840985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:07.840904    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cea476b-153c-413f-a8dd-de062323dfe5-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:08.512037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:08.512008    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"
Sep 09 13:17:08.512195 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:08.512043    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj" event={"ID":"3cea476b-153c-413f-a8dd-de062323dfe5","Type":"ContainerDied","Data":"fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4"}
Sep 09 13:17:08.512195 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:08.512091    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fc0feaf8cbadec4c3790e1d50cf2d19b45b09592f13ee993dc83c6ee89929ee4"
Sep 09 13:17:10.277549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.277478    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:10.315687 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.315652    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:17:10.316159 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.316119    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" podUID="34776766-362c-4fcc-a188-431b7781732a" containerName="payload-pre-processing" containerID="cri-o://3929464a18afb84cf1fd222e9b6691dbf3678075c62edbf1f995c4b73eca30d2" gracePeriod=30
Sep 09 13:17:10.376623 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.376586    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:10.457746 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.457715    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"]
Sep 09 13:17:10.462029 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.461982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:10.463991 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.463973    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-964720-pt2jj"]
Sep 09 13:17:10.517439 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.517407    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:10.527714 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.527649    2580 generic.go:356] "Generic (PLEG): container finished" podID="34776766-362c-4fcc-a188-431b7781732a" containerID="3929464a18afb84cf1fd222e9b6691dbf3678075c62edbf1f995c4b73eca30d2" exitCode=0
Sep 09 13:17:10.527714 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.527681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" event={"ID":"34776766-362c-4fcc-a188-431b7781732a","Type":"ContainerDied","Data":"3929464a18afb84cf1fd222e9b6691dbf3678075c62edbf1f995c4b73eca30d2"}
Sep 09 13:17:10.584139 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.584115    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:17:10.675256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.675223    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume\") pod \"34776766-362c-4fcc-a188-431b7781732a\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") "
Sep 09 13:17:10.675433 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.675310    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzbnw\" (UniqueName: \"kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw\") pod \"34776766-362c-4fcc-a188-431b7781732a\" (UID: \"34776766-362c-4fcc-a188-431b7781732a\") "
Sep 09 13:17:10.675623 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.675601    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "34776766-362c-4fcc-a188-431b7781732a" (UID: "34776766-362c-4fcc-a188-431b7781732a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:10.677346 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.677328    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw" (OuterVolumeSpecName: "kube-api-access-fzbnw") pod "34776766-362c-4fcc-a188-431b7781732a" (UID: "34776766-362c-4fcc-a188-431b7781732a"). InnerVolumeSpecName "kube-api-access-fzbnw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:10.776089 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.776056    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzbnw\" (UniqueName: \"kubernetes.io/projected/34776766-362c-4fcc-a188-431b7781732a-kube-api-access-fzbnw\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:10.776089 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.776084    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/34776766-362c-4fcc-a188-431b7781732a-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:10.832078 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:10.831982    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:11.536707 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:11.536671    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt" event={"ID":"34776766-362c-4fcc-a188-431b7781732a","Type":"ContainerDied","Data":"40c8180ef658a630f19c5c1f9a6cdf3b67f55e769dadaa934f0bad8b30311d45"}
Sep 09 13:17:11.537140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:11.536740    2580 scope.go:117] "RemoveContainer" containerID="3929464a18afb84cf1fd222e9b6691dbf3678075c62edbf1f995c4b73eca30d2"
Sep 09 13:17:11.537140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:11.536680    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"
Sep 09 13:17:11.558307 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:11.558281    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:17:11.561752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:11.561732    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-964720-78dc76858b-brxjt"]
Sep 09 13:17:12.453757 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:12.453721    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34776766-362c-4fcc-a188-431b7781732a" path="/var/lib/kubelet/pods/34776766-362c-4fcc-a188-431b7781732a/volumes"
Sep 09 13:17:12.454081 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:12.454061    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cea476b-153c-413f-a8dd-de062323dfe5" path="/var/lib/kubelet/pods/3cea476b-153c-413f-a8dd-de062323dfe5/volumes"
Sep 09 13:17:16.403092 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:16.403048    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:16.469312 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:16.469280    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:16.671934 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:16.671851    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:20.406132 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406095    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"]
Sep 09 13:17:20.406529 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406515    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34776766-362c-4fcc-a188-431b7781732a" containerName="payload-pre-processing"
Sep 09 13:17:20.406529 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406530    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="34776766-362c-4fcc-a188-431b7781732a" containerName="payload-pre-processing"
Sep 09 13:17:20.406626 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406546    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cea476b-153c-413f-a8dd-de062323dfe5" containerName="revoke-keys"
Sep 09 13:17:20.406626 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406553    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cea476b-153c-413f-a8dd-de062323dfe5" containerName="revoke-keys"
Sep 09 13:17:20.406696 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406640    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="34776766-362c-4fcc-a188-431b7781732a" containerName="payload-pre-processing"
Sep 09 13:17:20.406696 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.406647    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cea476b-153c-413f-a8dd-de062323dfe5" containerName="revoke-keys"
Sep 09 13:17:20.410785 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.410766    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:20.413290 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.413265    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:17:20.416955 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.416922    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"]
Sep 09 13:17:20.457235 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.457210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w\" (UID: \"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:20.516088 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.516057    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:20.558199 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.558167    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w\" (UID: \"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:20.558876 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.558855    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w\" (UID: \"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:20.725987 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:20.725912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:21.000236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:21.000201    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"]
Sep 09 13:17:21.001963 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:17:21.001941    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa90c5e_951d_46b6_a3f7_5b5ab1e30bf4.slice/crio-b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6 WatchSource:0}: Error finding container b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6: Status 404 returned error can't find the container with id b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6
Sep 09 13:17:21.602765 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:21.602680    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w" event={"ID":"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4","Type":"ContainerStarted","Data":"823d91de1653ddf4cde7b48865b121a68b83cd7c8409a4cb9aa39cb0d6587b62"}
Sep 09 13:17:21.602765 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:21.602728    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w" event={"ID":"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4","Type":"ContainerStarted","Data":"b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6"}
Sep 09 13:17:21.617649 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:21.617606    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w" podStartSLOduration=1.617590899 podStartE2EDuration="1.617590899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:17:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:17:21.615490025 +0000 UTC m=+1739.650737931" watchObservedRunningTime="2026-09-09 13:17:21.617590899 +0000 UTC m=+1739.652838805"
Sep 09 13:17:22.609922 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:22.609886    2580 generic.go:356] "Generic (PLEG): container finished" podID="aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" containerID="823d91de1653ddf4cde7b48865b121a68b83cd7c8409a4cb9aa39cb0d6587b62" exitCode=0
Sep 09 13:17:22.610317 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:22.609973    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w" event={"ID":"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4","Type":"ContainerDied","Data":"823d91de1653ddf4cde7b48865b121a68b83cd7c8409a4cb9aa39cb0d6587b62"}
Sep 09 13:17:22.831575 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:22.831538    2580 scope.go:117] "RemoveContainer" containerID="970326b88ffef4ebd48e9cc76d8e9cd8e31c7dad2f1b2cb65049d3c2bde85cec"
Sep 09 13:17:23.751999 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:23.751976    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:23.889790 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:23.889693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca\") pod \"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4\" (UID: \"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4\") "
Sep 09 13:17:23.890051 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:23.890025    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" (UID: "aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:23.990822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:23.990788    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:24.625173 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:24.625143    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"
Sep 09 13:17:24.625335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:24.625193    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w" event={"ID":"aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4","Type":"ContainerDied","Data":"b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6"}
Sep 09 13:17:24.625335 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:24.625246    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b54eda61a64aba7c6788ef27d28be8ef33f35fc5a9bb594dd47f1dcf5a4b23b6"
Sep 09 13:17:36.018011 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.017976    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:36.050013 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.049978    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:17:36.050232 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.050205    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" podUID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" containerName="payload-pre-processing" containerID="cri-o://d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed" gracePeriod=30
Sep 09 13:17:36.117371 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.117334    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:36.205240 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.205189    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"]
Sep 09 13:17:36.206461 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.206418    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:36.207905 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.207884    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-458b0f-hcg2w"]
Sep 09 13:17:36.259315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.259267    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:36.323620 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.323593    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:17:36.401124 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.401085    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7w8fs\" (UniqueName: \"kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs\") pod \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") "
Sep 09 13:17:36.401310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.401144    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume\") pod \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\" (UID: \"1a913dde-ba74-4b64-9d02-cdd7a9fa924e\") "
Sep 09 13:17:36.401566 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.401536    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a913dde-ba74-4b64-9d02-cdd7a9fa924e" (UID: "1a913dde-ba74-4b64-9d02-cdd7a9fa924e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:17:36.403166 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.403139    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs" (OuterVolumeSpecName: "kube-api-access-7w8fs") pod "1a913dde-ba74-4b64-9d02-cdd7a9fa924e" (UID: "1a913dde-ba74-4b64-9d02-cdd7a9fa924e"). InnerVolumeSpecName "kube-api-access-7w8fs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:36.453437 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.453406    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" path="/var/lib/kubelet/pods/aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4/volumes"
Sep 09 13:17:36.501873 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.501846    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7w8fs\" (UniqueName: \"kubernetes.io/projected/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-kube-api-access-7w8fs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:36.501873 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.501870    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a913dde-ba74-4b64-9d02-cdd7a9fa924e-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:36.590003 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.589903    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:36.702332 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.702298    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" containerID="d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed" exitCode=0
Sep 09 13:17:36.702549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.702339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"
Sep 09 13:17:36.702549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.702378    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" event={"ID":"1a913dde-ba74-4b64-9d02-cdd7a9fa924e","Type":"ContainerDied","Data":"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"}
Sep 09 13:17:36.702549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.702423    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr" event={"ID":"1a913dde-ba74-4b64-9d02-cdd7a9fa924e","Type":"ContainerDied","Data":"f26ee84464fcce044ef3c4fe06db7c91e7ed43bf9009e23471a234a6478f9021"}
Sep 09 13:17:36.702549 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.702443    2580 scope.go:117] "RemoveContainer" containerID="d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"
Sep 09 13:17:36.712942 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.712923    2580 scope.go:117] "RemoveContainer" containerID="d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"
Sep 09 13:17:36.713251 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:17:36.713227    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed\": container with ID starting with d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed not found: ID does not exist" containerID="d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"
Sep 09 13:17:36.713336 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.713263    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed"} err="failed to get container status \"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed\": rpc error: code = NotFound desc = could not find container \"d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed\": container with ID starting with d7af6dcc0081f32010f3d2face5276233314fd0e32c66d3a2122681aa80f68ed not found: ID does not exist"
Sep 09 13:17:36.724061 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.724018    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:17:36.729270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:36.729249    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-458b0f-84785bc4c4-qrqpr"]
Sep 09 13:17:38.454167 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:38.454135    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" path="/var/lib/kubelet/pods/1a913dde-ba74-4b64-9d02-cdd7a9fa924e/volumes"
Sep 09 13:17:41.578932 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:41.578891    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:17:41.673903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:41.673825    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:17:56.599586 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.599544    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" podUID="4f03c2de-04e6-4edc-a460-0141cd54e458" containerName="main" containerID="cri-o://985b00fae6c78fd3ed97edae7dd951244f95cdb371120e711b315e8aa679adf5" gracePeriod=45
Sep 09 13:17:56.831869 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.831839    2580 generic.go:356] "Generic (PLEG): container finished" podID="4f03c2de-04e6-4edc-a460-0141cd54e458" containerID="985b00fae6c78fd3ed97edae7dd951244f95cdb371120e711b315e8aa679adf5" exitCode=0
Sep 09 13:17:56.831983 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.831872    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" event={"ID":"4f03c2de-04e6-4edc-a460-0141cd54e458","Type":"ContainerDied","Data":"985b00fae6c78fd3ed97edae7dd951244f95cdb371120e711b315e8aa679adf5"}
Sep 09 13:17:56.847231 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.847213    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:17:56.878618 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878541    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.878618 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878609    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5m2vv\" (UniqueName: \"kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.878833 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878635    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.878833 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878803    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.878959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878846    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.878959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home" (OuterVolumeSpecName: "home") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:56.878959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.878884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir\") pod \"4f03c2de-04e6-4edc-a460-0141cd54e458\" (UID: \"4f03c2de-04e6-4edc-a460-0141cd54e458\") "
Sep 09 13:17:56.879125 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.879019    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache" (OuterVolumeSpecName: "model-cache") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:56.879240 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.879188    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:56.879310 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.879269    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-home\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:56.879368 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.879321    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-model-cache\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:56.880914 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.880886    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm" (OuterVolumeSpecName: "dshm") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:17:56.880914 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.880883    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv" (OuterVolumeSpecName: "kube-api-access-5m2vv") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "kube-api-access-5m2vv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:17:56.881187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.881168    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4f03c2de-04e6-4edc-a460-0141cd54e458" (UID: "4f03c2de-04e6-4edc-a460-0141cd54e458"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:17:56.979941 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.979900    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5m2vv\" (UniqueName: \"kubernetes.io/projected/4f03c2de-04e6-4edc-a460-0141cd54e458-kube-api-access-5m2vv\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:56.979941 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.979935    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-dshm\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:56.980158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.979952    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4f03c2de-04e6-4edc-a460-0141cd54e458-tmp-dir\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:56.980158 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:56.979963    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4f03c2de-04e6-4edc-a460-0141cd54e458-tls-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:17:57.840541 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:57.840495    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22" event={"ID":"4f03c2de-04e6-4edc-a460-0141cd54e458","Type":"ContainerDied","Data":"6c69225d3faa7438bd503da1dc546b37b6d2cb38caf88f99d4c4f0281b851156"}
Sep 09 13:17:57.840964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:57.840557    2580 scope.go:117] "RemoveContainer" containerID="985b00fae6c78fd3ed97edae7dd951244f95cdb371120e711b315e8aa679adf5"
Sep 09 13:17:57.840964 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:57.840516    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"
Sep 09 13:17:57.863798 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:57.863761    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:17:57.867833 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:57.867810    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-458b0f/ipp-route-458b0f-kserve-7f64c8848c-n6d22"]
Sep 09 13:17:58.453052 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:17:58.453020    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f03c2de-04e6-4edc-a460-0141cd54e458" path="/var/lib/kubelet/pods/4f03c2de-04e6-4edc-a460-0141cd54e458/volumes"
Sep 09 13:18:18.286067 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:18.285965    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:18.609476 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:18.609376    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:22.601808 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:22.601783    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:18:22.605771 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:22.605751    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:18:22.858021 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:22.857935    2580 scope.go:117] "RemoveContainer" containerID="423272c650007a48755c3ceeeceec9cfbb5017c900ed63e0eefd933da7f51db0"
Sep 09 13:18:23.986938 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:23.986899    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:24.087137 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.087103    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:24.277843 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.277756    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:24.662381 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.662347    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"]
Sep 09 13:18:24.663030 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663010    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4f03c2de-04e6-4edc-a460-0141cd54e458" containerName="main"
Sep 09 13:18:24.663114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663033    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4f03c2de-04e6-4edc-a460-0141cd54e458" containerName="main"
Sep 09 13:18:24.663114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663045    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" containerName="revoke-keys"
Sep 09 13:18:24.663114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663054    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" containerName="revoke-keys"
Sep 09 13:18:24.663114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663070    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" containerName="payload-pre-processing"
Sep 09 13:18:24.663114 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663078    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" containerName="payload-pre-processing"
Sep 09 13:18:24.663276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663162    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa90c5e-951d-46b6-a3f7-5b5ab1e30bf4" containerName="revoke-keys"
Sep 09 13:18:24.663276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663175    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4f03c2de-04e6-4edc-a460-0141cd54e458" containerName="main"
Sep 09 13:18:24.663276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.663184    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a913dde-ba74-4b64-9d02-cdd7a9fa924e" containerName="payload-pre-processing"
Sep 09 13:18:24.667814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.667799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:24.671767 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.670597    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:18:24.672669 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.672646    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"]
Sep 09 13:18:24.730900 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.730867    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd\" (UID: \"341d3bdd-e5b9-4c82-8261-6ab102064b3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:24.801746 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.801714    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:24.831951 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.831914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd\" (UID: \"341d3bdd-e5b9-4c82-8261-6ab102064b3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:24.832596 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.832573    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd\" (UID: \"341d3bdd-e5b9-4c82-8261-6ab102064b3f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:24.981493 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:24.981407    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:25.255696 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:25.255667    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"]
Sep 09 13:18:25.256771 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:18:25.256746    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod341d3bdd_e5b9_4c82_8261_6ab102064b3f.slice/crio-97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667 WatchSource:0}: Error finding container 97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667: Status 404 returned error can't find the container with id 97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667
Sep 09 13:18:26.026051 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:26.026003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd" event={"ID":"341d3bdd-e5b9-4c82-8261-6ab102064b3f","Type":"ContainerStarted","Data":"657fbce0a0b1941d21950d8c8c3ec98b9d4e74bfbb9ad7ce08b3f51a57f4469b"}
Sep 09 13:18:26.026236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:26.026076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd" event={"ID":"341d3bdd-e5b9-4c82-8261-6ab102064b3f","Type":"ContainerStarted","Data":"97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667"}
Sep 09 13:18:26.040896 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:26.040853    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd" podStartSLOduration=2.040838955 podStartE2EDuration="2.040838955s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:18:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:18:26.038470843 +0000 UTC m=+1804.073718752" watchObservedRunningTime="2026-09-09 13:18:26.040838955 +0000 UTC m=+1804.076086861"
Sep 09 13:18:27.033662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:27.033630    2580 generic.go:356] "Generic (PLEG): container finished" podID="341d3bdd-e5b9-4c82-8261-6ab102064b3f" containerID="657fbce0a0b1941d21950d8c8c3ec98b9d4e74bfbb9ad7ce08b3f51a57f4469b" exitCode=0
Sep 09 13:18:27.034099 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:27.033709    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd" event={"ID":"341d3bdd-e5b9-4c82-8261-6ab102064b3f","Type":"ContainerDied","Data":"657fbce0a0b1941d21950d8c8c3ec98b9d4e74bfbb9ad7ce08b3f51a57f4469b"}
Sep 09 13:18:28.173875 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:28.173852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:28.262214 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:28.262178    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca\") pod \"341d3bdd-e5b9-4c82-8261-6ab102064b3f\" (UID: \"341d3bdd-e5b9-4c82-8261-6ab102064b3f\") "
Sep 09 13:18:28.262574 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:28.262551    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "341d3bdd-e5b9-4c82-8261-6ab102064b3f" (UID: "341d3bdd-e5b9-4c82-8261-6ab102064b3f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:28.363766 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:28.363685    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/341d3bdd-e5b9-4c82-8261-6ab102064b3f-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:18:29.048111 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:29.048081    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"
Sep 09 13:18:29.048291 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:29.048119    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd" event={"ID":"341d3bdd-e5b9-4c82-8261-6ab102064b3f","Type":"ContainerDied","Data":"97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667"}
Sep 09 13:18:29.048291 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:29.048195    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97e7f8cc2f7e5ea93b4a4df60eea6f188f1ffa19a11dd8ced25b59bf51ed9667"
Sep 09 13:18:35.151643 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.151256    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:18:35.151643 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.151478    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" podUID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" containerName="payload-pre-processing" containerID="cri-o://48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4" gracePeriod=30
Sep 09 13:18:35.226419 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.226378    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:35.326539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.326481    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:35.344305 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.344279    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"]
Sep 09 13:18:35.346877 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.346849    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-ae6936-jmczd"]
Sep 09 13:18:35.409282 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.409224    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:18:35.426980 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.426866    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:35.527050 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.527012    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:35.533121 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.533098    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-447gq\" (UniqueName: \"kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq\") pod \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") "
Sep 09 13:18:35.533228 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.533151    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume\") pod \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\" (UID: \"86063bfb-9bcc-4302-bad9-1a9d513d5ea0\") "
Sep 09 13:18:35.533582 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.533556    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "86063bfb-9bcc-4302-bad9-1a9d513d5ea0" (UID: "86063bfb-9bcc-4302-bad9-1a9d513d5ea0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:35.535228 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.535198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq" (OuterVolumeSpecName: "kube-api-access-447gq") pod "86063bfb-9bcc-4302-bad9-1a9d513d5ea0" (UID: "86063bfb-9bcc-4302-bad9-1a9d513d5ea0"). InnerVolumeSpecName "kube-api-access-447gq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:18:35.633814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.633780    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-447gq\" (UniqueName: \"kubernetes.io/projected/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-kube-api-access-447gq\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:18:35.633814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.633811    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/86063bfb-9bcc-4302-bad9-1a9d513d5ea0-plugins-config-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:18:35.726256 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:35.726169    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:36.095734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.095570    2580 generic.go:356] "Generic (PLEG): container finished" podID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" containerID="48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4" exitCode=0
Sep 09 13:18:36.095734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.095631    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"
Sep 09 13:18:36.095734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.095711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" event={"ID":"86063bfb-9bcc-4302-bad9-1a9d513d5ea0","Type":"ContainerDied","Data":"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"}
Sep 09 13:18:36.096255 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.095788    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b" event={"ID":"86063bfb-9bcc-4302-bad9-1a9d513d5ea0","Type":"ContainerDied","Data":"22e08946b4854f794df19db87422428e24792e0a8c081c281b27aaa08a2a79b2"}
Sep 09 13:18:36.096255 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.095831    2580 scope.go:117] "RemoveContainer" containerID="48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"
Sep 09 13:18:36.106414 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.106398    2580 scope.go:117] "RemoveContainer" containerID="48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"
Sep 09 13:18:36.106744 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:18:36.106723    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4\": container with ID starting with 48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4 not found: ID does not exist" containerID="48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"
Sep 09 13:18:36.106822 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.106752    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4"} err="failed to get container status \"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4\": rpc error: code = NotFound desc = could not find container \"48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4\": container with ID starting with 48acd77c56354fec97c84b58c92bb6f5ce842ed6d5d649b1aab509f94dc866f4 not found: ID does not exist"
Sep 09 13:18:36.118808 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.118782    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:18:36.123715 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.123694    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-ae6936-8cff55c7d-2646b"]
Sep 09 13:18:36.453043 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.453004    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="341d3bdd-e5b9-4c82-8261-6ab102064b3f" path="/var/lib/kubelet/pods/341d3bdd-e5b9-4c82-8261-6ab102064b3f/volumes"
Sep 09 13:18:36.453415 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:36.453351    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" path="/var/lib/kubelet/pods/86063bfb-9bcc-4302-bad9-1a9d513d5ea0/volumes"
Sep 09 13:18:41.529922 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:41.529879    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:41.596210 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:41.596176    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:41.805901 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:41.805808    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:42.156668 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.156628    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"]
Sep 09 13:18:42.157116 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157096    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="341d3bdd-e5b9-4c82-8261-6ab102064b3f" containerName="revoke-keys"
Sep 09 13:18:42.157213 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157119    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="341d3bdd-e5b9-4c82-8261-6ab102064b3f" containerName="revoke-keys"
Sep 09 13:18:42.157213 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157141    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" containerName="payload-pre-processing"
Sep 09 13:18:42.157213 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157150    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" containerName="payload-pre-processing"
Sep 09 13:18:42.157359 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157258    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="341d3bdd-e5b9-4c82-8261-6ab102064b3f" containerName="revoke-keys"
Sep 09 13:18:42.157359 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.157273    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="86063bfb-9bcc-4302-bad9-1a9d513d5ea0" containerName="payload-pre-processing"
Sep 09 13:18:42.161842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.161822    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:42.164583 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.164560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-t5rn5\""
Sep 09 13:18:42.167234 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.167212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"]
Sep 09 13:18:42.195160 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.195121    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx\" (UID: \"b69b623f-b094-4723-9531-4fe92e1c412d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:42.296313 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.296280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx\" (UID: \"b69b623f-b094-4723-9531-4fe92e1c412d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:42.296980 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.296955    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx\" (UID: \"b69b623f-b094-4723-9531-4fe92e1c412d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:42.474668 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.474586    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:42.750204 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:42.750172    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"]
Sep 09 13:18:42.751266 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:18:42.751241    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69b623f_b094_4723_9531_4fe92e1c412d.slice/crio-79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1 WatchSource:0}: Error finding container 79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1: Status 404 returned error can't find the container with id 79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1
Sep 09 13:18:43.149519 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:43.149466    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx" event={"ID":"b69b623f-b094-4723-9531-4fe92e1c412d","Type":"ContainerStarted","Data":"87133451e2c8753ce555b7a3c6c13e84e7a285b03e5620d6b8963e37317fd7f1"}
Sep 09 13:18:43.149702 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:43.149537    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx" event={"ID":"b69b623f-b094-4723-9531-4fe92e1c412d","Type":"ContainerStarted","Data":"79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1"}
Sep 09 13:18:43.170258 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:43.170206    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx" podStartSLOduration=1.1701909480000001 podStartE2EDuration="1.170190948s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:18:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:18:43.167458174 +0000 UTC m=+1821.202706081" watchObservedRunningTime="2026-09-09 13:18:43.170190948 +0000 UTC m=+1821.205438855"
Sep 09 13:18:44.157924 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:44.157889    2580 generic.go:356] "Generic (PLEG): container finished" podID="b69b623f-b094-4723-9531-4fe92e1c412d" containerID="87133451e2c8753ce555b7a3c6c13e84e7a285b03e5620d6b8963e37317fd7f1" exitCode=0
Sep 09 13:18:44.158319 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:44.157977    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx" event={"ID":"b69b623f-b094-4723-9531-4fe92e1c412d","Type":"ContainerDied","Data":"87133451e2c8753ce555b7a3c6c13e84e7a285b03e5620d6b8963e37317fd7f1"}
Sep 09 13:18:45.294440 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:45.294419    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:45.320585 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:45.320555    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca\") pod \"b69b623f-b094-4723-9531-4fe92e1c412d\" (UID: \"b69b623f-b094-4723-9531-4fe92e1c412d\") "
Sep 09 13:18:45.320920 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:45.320896    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b69b623f-b094-4723-9531-4fe92e1c412d" (UID: "b69b623f-b094-4723-9531-4fe92e1c412d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:18:45.422319 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:45.422208    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b69b623f-b094-4723-9531-4fe92e1c412d-maas-api-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:18:46.171766 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:46.171734    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"
Sep 09 13:18:46.171766 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:46.171752    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx" event={"ID":"b69b623f-b094-4723-9531-4fe92e1c412d","Type":"ContainerDied","Data":"79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1"}
Sep 09 13:18:46.171986 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:46.171796    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="79ceb7a8d4a42c97e5e9f28fcc96282c006523980277b8e1050d6e98540810e1"
Sep 09 13:18:52.706077 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.705977    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:52.712719 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.712689    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"]
Sep 09 13:18:52.716732 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.716706    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w3-ada1ae-29815995-6p875"]
Sep 09 13:18:52.778169 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.778138    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:52.880928 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.880892    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:52.898236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.898207    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"]
Sep 09 13:18:52.900448 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.900429    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w3-ada1ae-9gnrx"]
Sep 09 13:18:52.943841 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:52.943803    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:53.296140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:53.296100    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:18:54.452775 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:54.452738    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d0466ab-1616-4a2b-bc75-c195f44b92fa" path="/var/lib/kubelet/pods/7d0466ab-1616-4a2b-bc75-c195f44b92fa/volumes"
Sep 09 13:18:54.453159 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:54.453041    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b69b623f-b094-4723-9531-4fe92e1c412d" path="/var/lib/kubelet/pods/b69b623f-b094-4723-9531-4fe92e1c412d/volumes"
Sep 09 13:18:59.132823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:18:59.132781    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:00.565925 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:00.565880    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:10.186700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.186662    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"]
Sep 09 13:19:10.192694 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.192665    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w3-650a99-29815995-5hrh4"]
Sep 09 13:19:10.194418 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.194387    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:10.284396 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.284360    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:10.380091 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.380052    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:10.442547 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.442426    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:10.452752 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.452726    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb00ef94-84e6-48f9-a396-b73fd9b01f09" path="/var/lib/kubelet/pods/eb00ef94-84e6-48f9-a396-b73fd9b01f09/volumes"
Sep 09 13:19:10.779711 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:10.779623    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:16.930606 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:16.930571    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:16.999342 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:16.999304    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:17.070737 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:17.070703    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:19.709970 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:19.709927    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 13:19:19.710559 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:19.710162    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-885499b4f-xq4z5" podUID="8feb91b2-4adb-442b-8333-8da3c2329680" containerName="manager" containerID="cri-o://a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b" gracePeriod=30
Sep 09 13:19:19.972140 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:19.972074    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 13:19:20.156235 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.156198    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert\") pod \"8feb91b2-4adb-442b-8333-8da3c2329680\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") "
Sep 09 13:19:20.156446 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.156277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs\") pod \"8feb91b2-4adb-442b-8333-8da3c2329680\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") "
Sep 09 13:19:20.156446 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.156342    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v9q8r\" (UniqueName: \"kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r\") pod \"8feb91b2-4adb-442b-8333-8da3c2329680\" (UID: \"8feb91b2-4adb-442b-8333-8da3c2329680\") "
Sep 09 13:19:20.158427 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.158394    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8feb91b2-4adb-442b-8333-8da3c2329680" (UID: "8feb91b2-4adb-442b-8333-8da3c2329680"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:19:20.158562 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.158475    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8feb91b2-4adb-442b-8333-8da3c2329680" (UID: "8feb91b2-4adb-442b-8333-8da3c2329680"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:19:20.158562 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.158495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r" (OuterVolumeSpecName: "kube-api-access-v9q8r") pod "8feb91b2-4adb-442b-8333-8da3c2329680" (UID: "8feb91b2-4adb-442b-8333-8da3c2329680"). InnerVolumeSpecName "kube-api-access-v9q8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:19:20.257098 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.257013    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:19:20.257098 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.257043    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v9q8r\" (UniqueName: \"kubernetes.io/projected/8feb91b2-4adb-442b-8333-8da3c2329680-kube-api-access-v9q8r\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:19:20.257098 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.257054    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8feb91b2-4adb-442b-8333-8da3c2329680-webhook-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:19:20.387625 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.387586    2580 generic.go:356] "Generic (PLEG): container finished" podID="8feb91b2-4adb-442b-8333-8da3c2329680" containerID="a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b" exitCode=0
Sep 09 13:19:20.387796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.387641    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-xq4z5"
Sep 09 13:19:20.387796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.387678    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-xq4z5" event={"ID":"8feb91b2-4adb-442b-8333-8da3c2329680","Type":"ContainerDied","Data":"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"}
Sep 09 13:19:20.387796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.387736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-xq4z5" event={"ID":"8feb91b2-4adb-442b-8333-8da3c2329680","Type":"ContainerDied","Data":"032309c78cd8090c73e1f82dc20ade2f50dabcfe65c04a8b324a31c60717afc7"}
Sep 09 13:19:20.387796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.387758    2580 scope.go:117] "RemoveContainer" containerID="a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"
Sep 09 13:19:20.406009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.405988    2580 scope.go:117] "RemoveContainer" containerID="a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"
Sep 09 13:19:20.406275 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:19:20.406258    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b\": container with ID starting with a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b not found: ID does not exist" containerID="a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"
Sep 09 13:19:20.406318 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.406284    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b"} err="failed to get container status \"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b\": rpc error: code = NotFound desc = could not find container \"a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b\": container with ID starting with a14b27b287dcb6f420aa313809f3ff25b638a41b0018f78244a2e32498194e0b not found: ID does not exist"
Sep 09 13:19:20.412475 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.412452    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 13:19:20.415757 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.415735    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-885499b4f-xq4z5"]
Sep 09 13:19:20.453207 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:20.453179    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8feb91b2-4adb-442b-8333-8da3c2329680" path="/var/lib/kubelet/pods/8feb91b2-4adb-442b-8333-8da3c2329680/volumes"
Sep 09 13:19:22.891176 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:22.891142    2580 scope.go:117] "RemoveContainer" containerID="7b7f557c0f557a002dc63e1dc4f3de1130ba2a7247f67652323c0e9639ae08bc"
Sep 09 13:19:43.941065 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:43.941030    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:19:43.996828 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:19:43.996795    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:20:21.227018 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.226983    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227364    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b69b623f-b094-4723-9531-4fe92e1c412d" containerName="revoke-keys"
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227376    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b69b623f-b094-4723-9531-4fe92e1c412d" containerName="revoke-keys"
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227393    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8feb91b2-4adb-442b-8333-8da3c2329680" containerName="manager"
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227399    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8feb91b2-4adb-442b-8333-8da3c2329680" containerName="manager"
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227452    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8feb91b2-4adb-442b-8333-8da3c2329680" containerName="manager"
Sep 09 13:20:21.227494 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.227462    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b69b623f-b094-4723-9531-4fe92e1c412d" containerName="revoke-keys"
Sep 09 13:20:21.230429 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.230412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.233554 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.233530    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 13:20:21.233554 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.233544    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:20:21.233750 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.233560    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 13:20:21.233750 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.233530    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g2wjn\""
Sep 09 13:20:21.238608 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.238589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:21.276263 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.276229    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfkrt\" (UniqueName: \"kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.276396 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.276298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.276396 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.276338    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.376815 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.376791    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfkrt\" (UniqueName: \"kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.376943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.376846    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.376943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.376884    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.379295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.379271    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.379392 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.379317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.389626 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.389607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfkrt\" (UniqueName: \"kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt\") pod \"maas-controller-885499b4f-7w5xn\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") " pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.541475 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.541400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:21.810952 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.810930    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:21.811957 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:20:21.811933    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb679b904_487c_48cf_995b_c990a6856e35.slice/crio-67aca831db1651662fc2d2b0827f75443233d2888392b3d3b46e02b9e5bc24fa WatchSource:0}: Error finding container 67aca831db1651662fc2d2b0827f75443233d2888392b3d3b46e02b9e5bc24fa: Status 404 returned error can't find the container with id 67aca831db1651662fc2d2b0827f75443233d2888392b3d3b46e02b9e5bc24fa
Sep 09 13:20:21.813135 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:21.813117    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 13:20:22.691823 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.691799    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:20:22.781959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.781924    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-7w5xn" event={"ID":"b679b904-487c-48cf-995b-c990a6856e35","Type":"ContainerStarted","Data":"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"}
Sep 09 13:20:22.782111 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.781985    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-7w5xn" event={"ID":"b679b904-487c-48cf-995b-c990a6856e35","Type":"ContainerStarted","Data":"67aca831db1651662fc2d2b0827f75443233d2888392b3d3b46e02b9e5bc24fa"}
Sep 09 13:20:22.782111 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.782042    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:22.800629 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.800588    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-885499b4f-7w5xn" podStartSLOduration=0.924285273 podStartE2EDuration="1.800576205s" totalImagesPullingTime="876.290932ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:20:22.797464304 +0000 UTC m=+1920.832712210" watchObservedRunningTime="2026-09-09 13:20:22.800576205 +0000 UTC m=+1920.835824113"
Sep 09 13:20:22.924665 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:22.924634    2580 scope.go:117] "RemoveContainer" containerID="582f7a1a2f15779491e92b20884e260278b0bd3342b8a16f75ea1221246ce63d"
Sep 09 13:20:33.790899 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:33.790868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:43.019303 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.019255    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:20:43.262245 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.262208    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:20:43.370197 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.370117    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:43.370375 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.370349    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-885499b4f-7w5xn" podUID="b679b904-487c-48cf-995b-c990a6856e35" containerName="manager" containerID="cri-o://a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df" gracePeriod=30
Sep 09 13:20:43.617796 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.617775    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:43.772629 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.772597    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs\") pod \"b679b904-487c-48cf-995b-c990a6856e35\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") "
Sep 09 13:20:43.772786 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.772680    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfkrt\" (UniqueName: \"kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt\") pod \"b679b904-487c-48cf-995b-c990a6856e35\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") "
Sep 09 13:20:43.772854 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.772833    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert\") pod \"b679b904-487c-48cf-995b-c990a6856e35\" (UID: \"b679b904-487c-48cf-995b-c990a6856e35\") "
Sep 09 13:20:43.774730 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.774700    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt" (OuterVolumeSpecName: "kube-api-access-wfkrt") pod "b679b904-487c-48cf-995b-c990a6856e35" (UID: "b679b904-487c-48cf-995b-c990a6856e35"). InnerVolumeSpecName "kube-api-access-wfkrt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:20:43.774919 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.774805    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b679b904-487c-48cf-995b-c990a6856e35" (UID: "b679b904-487c-48cf-995b-c990a6856e35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:20:43.774919 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.774829    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "b679b904-487c-48cf-995b-c990a6856e35" (UID: "b679b904-487c-48cf-995b-c990a6856e35"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:20:43.873903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.873879    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfkrt\" (UniqueName: \"kubernetes.io/projected/b679b904-487c-48cf-995b-c990a6856e35-kube-api-access-wfkrt\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:20:43.873903 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.873900    2580 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-webhook-cert\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:20:43.874067 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.873910    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b679b904-487c-48cf-995b-c990a6856e35-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:20:43.909184 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.909152    2580 generic.go:356] "Generic (PLEG): container finished" podID="b679b904-487c-48cf-995b-c990a6856e35" containerID="a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df" exitCode=0
Sep 09 13:20:43.909295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.909199    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-7w5xn"
Sep 09 13:20:43.909295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.909230    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-7w5xn" event={"ID":"b679b904-487c-48cf-995b-c990a6856e35","Type":"ContainerDied","Data":"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"}
Sep 09 13:20:43.909295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.909288    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-7w5xn" event={"ID":"b679b904-487c-48cf-995b-c990a6856e35","Type":"ContainerDied","Data":"67aca831db1651662fc2d2b0827f75443233d2888392b3d3b46e02b9e5bc24fa"}
Sep 09 13:20:43.909400 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.909311    2580 scope.go:117] "RemoveContainer" containerID="a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"
Sep 09 13:20:43.919814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.919792    2580 scope.go:117] "RemoveContainer" containerID="a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"
Sep 09 13:20:43.920086 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:20:43.920068    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df\": container with ID starting with a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df not found: ID does not exist" containerID="a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"
Sep 09 13:20:43.920156 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.920099    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df"} err="failed to get container status \"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df\": rpc error: code = NotFound desc = could not find container \"a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df\": container with ID starting with a03e8840932a03a0d1934e4cb09de7d187598162a2c322e0ec9a399f8d63a7df not found: ID does not exist"
Sep 09 13:20:43.935194 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.935160    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:43.938875 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:43.938845    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-885499b4f-7w5xn"]
Sep 09 13:20:44.453147 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.453114    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b679b904-487c-48cf-995b-c990a6856e35" path="/var/lib/kubelet/pods/b679b904-487c-48cf-995b-c990a6856e35/volumes"
Sep 09 13:20:44.675345 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.675313    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:20:44.791810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.791739    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-885499b4f-drpss"]
Sep 09 13:20:44.792169 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.792155    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b679b904-487c-48cf-995b-c990a6856e35" containerName="manager"
Sep 09 13:20:44.792211 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.792170    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b679b904-487c-48cf-995b-c990a6856e35" containerName="manager"
Sep 09 13:20:44.792247 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.792240    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b679b904-487c-48cf-995b-c990a6856e35" containerName="manager"
Sep 09 13:20:44.796748 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.796729    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.799896 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.799870    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-g2wjn\""
Sep 09 13:20:44.800028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.799897    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 13:20:44.800028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.799940    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 13:20:44.800028 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.799951    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 13:20:44.805956 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.805938    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-drpss"]
Sep 09 13:20:44.882671 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.882639    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-webhook-cert\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.882831 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.882678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxlbs\" (UniqueName: \"kubernetes.io/projected/1da7c2b9-c716-417e-9eb2-639f89a2948f-kube-api-access-gxlbs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.882831 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.882764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-metrics-certs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.983397 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.983365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-webhook-cert\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.983592 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.983404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxlbs\" (UniqueName: \"kubernetes.io/projected/1da7c2b9-c716-417e-9eb2-639f89a2948f-kube-api-access-gxlbs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.983592 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.983470    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-metrics-certs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.985943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.985922    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-metrics-certs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.986037 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.986020    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1da7c2b9-c716-417e-9eb2-639f89a2948f-webhook-cert\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:44.990955 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:44.990934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxlbs\" (UniqueName: \"kubernetes.io/projected/1da7c2b9-c716-417e-9eb2-639f89a2948f-kube-api-access-gxlbs\") pod \"maas-controller-885499b4f-drpss\" (UID: \"1da7c2b9-c716-417e-9eb2-639f89a2948f\") " pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:45.108486 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:45.108414    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:45.382349 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:45.382319    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-885499b4f-drpss"]
Sep 09 13:20:45.384444 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:20:45.384416    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1da7c2b9_c716_417e_9eb2_639f89a2948f.slice/crio-7df7f95c2ac535289ee199748b6f887ddb3082374063a144b3f62f9240b2e527 WatchSource:0}: Error finding container 7df7f95c2ac535289ee199748b6f887ddb3082374063a144b3f62f9240b2e527: Status 404 returned error can't find the container with id 7df7f95c2ac535289ee199748b6f887ddb3082374063a144b3f62f9240b2e527
Sep 09 13:20:45.923970 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:45.923943    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-drpss" event={"ID":"1da7c2b9-c716-417e-9eb2-639f89a2948f","Type":"ContainerStarted","Data":"7df7f95c2ac535289ee199748b6f887ddb3082374063a144b3f62f9240b2e527"}
Sep 09 13:20:46.931698 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:46.931663    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-885499b4f-drpss" event={"ID":"1da7c2b9-c716-417e-9eb2-639f89a2948f","Type":"ContainerStarted","Data":"aa0802aff1cdbbc5add372615eef85bd1388d221122afec4a27ef4169fb9111b"}
Sep 09 13:20:46.932082 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:46.931721    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:20:46.948091 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:46.948041    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-885499b4f-drpss" podStartSLOduration=2.4407159370000002 podStartE2EDuration="2.948023929s" totalImagesPullingTime="507.307992ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:20:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:20:46.947336548 +0000 UTC m=+1944.982584456" watchObservedRunningTime="2026-09-09 13:20:46.948023929 +0000 UTC m=+1944.983271835"
Sep 09 13:20:57.940418 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:20:57.940387    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-885499b4f-drpss"
Sep 09 13:21:17.285004 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:17.284925    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:21:17.388850 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:17.388817    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:21:22.946916 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.946887    2580 scope.go:117] "RemoveContainer" containerID="4e28a2668a3d1c6615b00243e522f2e3fff0d546be3c2d3255897fb00cb9f929"
Sep 09 13:21:22.956045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.956020    2580 scope.go:117] "RemoveContainer" containerID="917fe9fde3e4957feeafc5d4f31d542d3b89de44a080fbcf7f186ee8091004ae"
Sep 09 13:21:22.964015 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.963992    2580 scope.go:117] "RemoveContainer" containerID="1dab634c0fbabd174fd6e94a5845831f456355285bae1246dc6d303624b4a3df"
Sep 09 13:21:22.971541 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.971520    2580 scope.go:117] "RemoveContainer" containerID="ae87b9a481a05adc613a060a1d0fe5d476d2b065fd1fba1be4d6b35c9b01c8e6"
Sep 09 13:21:22.979092 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.979074    2580 scope.go:117] "RemoveContainer" containerID="04847cb13c11946b16b1e5993cc932b482740f0d8026a91880c8c0ff0b5fe1e7"
Sep 09 13:21:22.986527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:22.986497    2580 scope.go:117] "RemoveContainer" containerID="8ed252c580c6e4fb9bd77ab283b2e44a67ba444b045616451c63fdb865e321d7"
Sep 09 13:21:50.159160 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:50.159116    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 13:21:50.159810 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:50.159359    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" podUID="467942ca-997c-478a-8b4e-1a7c7186cb41" containerName="manager" containerID="cri-o://96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693" gracePeriod=10
Sep 09 13:21:53.009006 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.008978    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 13:21:53.144521 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.144470    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h44p\" (UniqueName: \"kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p\") pod \"467942ca-997c-478a-8b4e-1a7c7186cb41\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") "
Sep 09 13:21:53.144684 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.144544    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume\") pod \"467942ca-997c-478a-8b4e-1a7c7186cb41\" (UID: \"467942ca-997c-478a-8b4e-1a7c7186cb41\") "
Sep 09 13:21:53.144953 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.144930    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "467942ca-997c-478a-8b4e-1a7c7186cb41" (UID: "467942ca-997c-478a-8b4e-1a7c7186cb41"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 13:21:53.146546 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.146520    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p" (OuterVolumeSpecName: "kube-api-access-4h44p") pod "467942ca-997c-478a-8b4e-1a7c7186cb41" (UID: "467942ca-997c-478a-8b4e-1a7c7186cb41"). InnerVolumeSpecName "kube-api-access-4h44p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:21:53.245264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.245227    2580 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/467942ca-997c-478a-8b4e-1a7c7186cb41-extensions-socket-volume\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:21:53.245264 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.245262    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h44p\" (UniqueName: \"kubernetes.io/projected/467942ca-997c-478a-8b4e-1a7c7186cb41-kube-api-access-4h44p\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:21:53.336723 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.336688    2580 generic.go:356] "Generic (PLEG): container finished" podID="467942ca-997c-478a-8b4e-1a7c7186cb41" containerID="96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693" exitCode=0
Sep 09 13:21:53.336868 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.336743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"
Sep 09 13:21:53.336868 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.336767    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" event={"ID":"467942ca-997c-478a-8b4e-1a7c7186cb41","Type":"ContainerDied","Data":"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"}
Sep 09 13:21:53.336868 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.336839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26" event={"ID":"467942ca-997c-478a-8b4e-1a7c7186cb41","Type":"ContainerDied","Data":"a08f480983980d163b118471c7d1b88ca2bfe06a233228f4442606c94ab854c4"}
Sep 09 13:21:53.336868 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.336861    2580 scope.go:117] "RemoveContainer" containerID="96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"
Sep 09 13:21:53.346628 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.346614    2580 scope.go:117] "RemoveContainer" containerID="96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"
Sep 09 13:21:53.346857 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:21:53.346838    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693\": container with ID starting with 96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693 not found: ID does not exist" containerID="96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"
Sep 09 13:21:53.346915 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.346869    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693"} err="failed to get container status \"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693\": rpc error: code = NotFound desc = could not find container \"96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693\": container with ID starting with 96b62acb32f22f07d90bbb88262106325999c355c610441e9f59eca89a39e693 not found: ID does not exist"
Sep 09 13:21:53.359224 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.359198    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 13:21:53.364004 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:53.363984    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-8dt26"]
Sep 09 13:21:54.453345 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:21:54.453311    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467942ca-997c-478a-8b4e-1a7c7186cb41" path="/var/lib/kubelet/pods/467942ca-997c-478a-8b4e-1a7c7186cb41/volumes"
Sep 09 13:22:50.945921 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.945888    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"]
Sep 09 13:22:50.946526 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.946316    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467942ca-997c-478a-8b4e-1a7c7186cb41" containerName="manager"
Sep 09 13:22:50.946526 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.946332    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="467942ca-997c-478a-8b4e-1a7c7186cb41" containerName="manager"
Sep 09 13:22:50.946526 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.946407    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="467942ca-997c-478a-8b4e-1a7c7186cb41" containerName="manager"
Sep 09 13:22:50.949543 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.949525    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:50.951945 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.951925    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lfmkw\""
Sep 09 13:22:50.962902 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:50.962881    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"]
Sep 09 13:22:51.020956 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.020923    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwdpw\" (UniqueName: \"kubernetes.io/projected/e2fba6a9-8411-4803-8b9b-703c17f193dc-kube-api-access-lwdpw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.021093 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.020990    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fba6a9-8411-4803-8b9b-703c17f193dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.122434 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.122392    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwdpw\" (UniqueName: \"kubernetes.io/projected/e2fba6a9-8411-4803-8b9b-703c17f193dc-kube-api-access-lwdpw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.122662 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.122453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fba6a9-8411-4803-8b9b-703c17f193dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.122842 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.122822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e2fba6a9-8411-4803-8b9b-703c17f193dc-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.130641 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.130616    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwdpw\" (UniqueName: \"kubernetes.io/projected/e2fba6a9-8411-4803-8b9b-703c17f193dc-kube-api-access-lwdpw\") pod \"kuadrant-operator-controller-manager-5769f99d9f-dcggn\" (UID: \"e2fba6a9-8411-4803-8b9b-703c17f193dc\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.259535 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.259423    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.534619 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.534594    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"]
Sep 09 13:22:51.536538 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:22:51.536481    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2fba6a9_8411_4803_8b9b_703c17f193dc.slice/crio-53079f15dfd73d556e2dc81fd1480730821d5c2fbd66e99abd3e2380356382a1 WatchSource:0}: Error finding container 53079f15dfd73d556e2dc81fd1480730821d5c2fbd66e99abd3e2380356382a1: Status 404 returned error can't find the container with id 53079f15dfd73d556e2dc81fd1480730821d5c2fbd66e99abd3e2380356382a1
Sep 09 13:22:51.677389 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.677356    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn" event={"ID":"e2fba6a9-8411-4803-8b9b-703c17f193dc","Type":"ContainerStarted","Data":"812dadd0f61823263e461d6eb70502aa03c9f03feddf6d334ad747cf57d96054"}
Sep 09 13:22:51.677581 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.677415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn" event={"ID":"e2fba6a9-8411-4803-8b9b-703c17f193dc","Type":"ContainerStarted","Data":"53079f15dfd73d556e2dc81fd1480730821d5c2fbd66e99abd3e2380356382a1"}
Sep 09 13:22:51.677581 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.677461    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:22:51.699788 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:22:51.699743    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn" podStartSLOduration=1.699729278 podStartE2EDuration="1.699729278s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:22:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:22:51.697341824 +0000 UTC m=+2069.732589730" watchObservedRunningTime="2026-09-09 13:22:51.699729278 +0000 UTC m=+2069.734977185"
Sep 09 13:23:02.685236 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:02.685203    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-dcggn"
Sep 09 13:23:22.642943 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:22.642909    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:23:22.648397 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:22.648373    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:23:23.048483 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:23.048387    2580 scope.go:117] "RemoveContainer" containerID="823d91de1653ddf4cde7b48865b121a68b83cd7c8409a4cb9aa39cb0d6587b62"
Sep 09 13:23:23.060219 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:23.060175    2580 scope.go:117] "RemoveContainer" containerID="007e9e9fd477640d3c76707031e4fc2c0e2a8893199f0120c6aace8854ad2dda"
Sep 09 13:23:31.019939 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:31.019893    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:23:35.874478 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:35.874439    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:23:57.451900 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:57.451860    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:23:59.073933 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:23:59.073896    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:24:30.477315 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:30.477183    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 13:24:30.477957 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:30.477496    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerName="maas-api" containerID="cri-o://baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076" gracePeriod=30
Sep 09 13:24:30.742618 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:30.742492    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.60:8443/health\": dial tcp 10.132.0.60:8443: connect: connection refused"
Sep 09 13:24:31.724913 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.724886    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 13:24:31.800074 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.800007    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls\") pod \"6b8042c8-4b1e-4035-9c4e-619bf105b659\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") "
Sep 09 13:24:31.800074 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.800067    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs\") pod \"6b8042c8-4b1e-4035-9c4e-619bf105b659\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") "
Sep 09 13:24:31.800281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.800118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca\") pod \"6b8042c8-4b1e-4035-9c4e-619bf105b659\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") "
Sep 09 13:24:31.800281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.800135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7zs2q\" (UniqueName: \"kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q\") pod \"6b8042c8-4b1e-4035-9c4e-619bf105b659\" (UID: \"6b8042c8-4b1e-4035-9c4e-619bf105b659\") "
Sep 09 13:24:31.800433 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.800406    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b8042c8-4b1e-4035-9c4e-619bf105b659" (UID: "6b8042c8-4b1e-4035-9c4e-619bf105b659"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 13:24:31.802071 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.802042    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b8042c8-4b1e-4035-9c4e-619bf105b659" (UID: "6b8042c8-4b1e-4035-9c4e-619bf105b659"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:24:31.802175 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.802102    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q" (OuterVolumeSpecName: "kube-api-access-7zs2q") pod "6b8042c8-4b1e-4035-9c4e-619bf105b659" (UID: "6b8042c8-4b1e-4035-9c4e-619bf105b659"). InnerVolumeSpecName "kube-api-access-7zs2q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 13:24:31.802175 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.802124    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b8042c8-4b1e-4035-9c4e-619bf105b659" (UID: "6b8042c8-4b1e-4035-9c4e-619bf105b659"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 13:24:31.901234 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.901199    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-metrics-certs\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:24:31.901234 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.901227    2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b8042c8-4b1e-4035-9c4e-619bf105b659-openshift-service-ca\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:24:31.901234 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.901237    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7zs2q\" (UniqueName: \"kubernetes.io/projected/6b8042c8-4b1e-4035-9c4e-619bf105b659-kube-api-access-7zs2q\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:24:31.901453 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:31.901245    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b8042c8-4b1e-4035-9c4e-619bf105b659-maas-api-tls\") on node \"ip-10-0-137-1.ec2.internal\" DevicePath \"\""
Sep 09 13:24:32.256078 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.256046    2580 generic.go:356] "Generic (PLEG): container finished" podID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerID="baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076" exitCode=0
Sep 09 13:24:32.256270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.256090    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"
Sep 09 13:24:32.256270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.256138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" event={"ID":"6b8042c8-4b1e-4035-9c4e-619bf105b659","Type":"ContainerDied","Data":"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"}
Sep 09 13:24:32.256270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.256227    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6" event={"ID":"6b8042c8-4b1e-4035-9c4e-619bf105b659","Type":"ContainerDied","Data":"bea89aaa117d582fb57b253247e62b697e1f211205c066b2eafcc4041f129cf8"}
Sep 09 13:24:32.256270 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.256255    2580 scope.go:117] "RemoveContainer" containerID="baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"
Sep 09 13:24:32.267255 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.267238    2580 scope.go:117] "RemoveContainer" containerID="baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"
Sep 09 13:24:32.267558 ip-10-0-137-1 kubenswrapper[2580]: E0909 13:24:32.267540    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076\": container with ID starting with baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076 not found: ID does not exist" containerID="baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"
Sep 09 13:24:32.267612 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.267565    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076"} err="failed to get container status \"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076\": rpc error: code = NotFound desc = could not find container \"baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076\": container with ID starting with baef00bec9d07ef1084a619b24a50b9fa88ea26caaca2f2afc7b2f3564284076 not found: ID does not exist"
Sep 09 13:24:32.279033 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.279007    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 13:24:32.283427 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.283406    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5dc88f4c96-78ft6"]
Sep 09 13:24:32.453353 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:32.453320    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" path="/var/lib/kubelet/pods/6b8042c8-4b1e-4035-9c4e-619bf105b659/volumes"
Sep 09 13:24:45.459920 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:45.459882    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56f8459dc5-m7glz_daa75492-94a7-46f2-b9e5-fa8df4e3c09d/authorino/0.log"
Sep 09 13:24:51.178849 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:51.178816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7gzcx_0d60c981-2889-4148-a482-9588c452044e/manager/0.log"
Sep 09 13:24:51.312533 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:51.312487    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9ppks_8bf8f3b2-e153-424b-9805-35ab0c027117/manager/0.log"
Sep 09 13:24:51.446277 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:51.446190    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-885499b4f-drpss_1da7c2b9-c716-417e-9eb2-639f89a2948f/manager/0.log"
Sep 09 13:24:52.125400 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:52.125370    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-dxwnv_ec32ce95-8011-4a26-b5c1-3b9e3138eeb5/manager/0.log"
Sep 09 13:24:52.132452 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:52.132427    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-dxwnv_ec32ce95-8011-4a26-b5c1-3b9e3138eeb5/copy-manifests/0.log"
Sep 09 13:24:52.977751 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:52.977717    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-594dz_e6ce55eb-699a-4893-ac5e-348344c63fc7/postgres/0.log"
Sep 09 13:24:53.797584 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:53.797556    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56f8459dc5-m7glz_daa75492-94a7-46f2-b9e5-fa8df4e3c09d/authorino/0.log"
Sep 09 13:24:53.918829 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:53.918797    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-pmb8j_9e0d4df1-2852-4075-a8a8-0280475be7f3/manager/0.log"
Sep 09 13:24:54.039321 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:54.039299    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5tkgp_f5f610ea-dba0-4363-8f43-1f345023176a/manager/0.log"
Sep 09 13:24:54.305249 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:54.305219    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-dcggn_e2fba6a9-8411-4803-8b9b-703c17f193dc/manager/0.log"
Sep 09 13:24:54.560005 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:54.559928    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-4v6zj_d60e23d2-c572-4c93-ab81-59f477b7e4b2/manager/0.log"
Sep 09 13:24:55.286558 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:55.286519    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-768599446d-wl78j_a6934b57-ea63-4e20-8245-2a833e114f76/kube-auth-proxy/0.log"
Sep 09 13:24:55.684241 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:55.684205    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:24:55.802799 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:55.802776    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b548b666-qdt2z_e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3/router/0.log"
Sep 09 13:24:56.406826 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:56.406793    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-qvgrv_61cae3b7-30fb-433e-8d49-3ec4e1e97c3c/main/0.log"
Sep 09 13:24:56.646005 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:24:56.645966    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f4rphm_62a4c5d5-627c-433e-9b47-653a51707824/main/0.log"
Sep 09 13:25:02.668422 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.668382    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sv79w/must-gather-d96xj"]
Sep 09 13:25:02.669009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.668796    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerName="maas-api"
Sep 09 13:25:02.669009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.668809    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerName="maas-api"
Sep 09 13:25:02.669009 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.668913    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b8042c8-4b1e-4035-9c4e-619bf105b659" containerName="maas-api"
Sep 09 13:25:02.671915 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.671894    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.674605 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.674578    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-sv79w\"/\"default-dockercfg-d6hxv\""
Sep 09 13:25:02.675438 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.675416    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sv79w\"/\"kube-root-ca.crt\""
Sep 09 13:25:02.675583 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.675445    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-sv79w\"/\"openshift-service-ca.crt\""
Sep 09 13:25:02.690268 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.690245    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/must-gather-d96xj"]
Sep 09 13:25:02.857930 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.857898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbj62\" (UniqueName: \"kubernetes.io/projected/b24175b8-d015-439b-8c61-7e8c0072d210-kube-api-access-bbj62\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.858094 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.857941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b24175b8-d015-439b-8c61-7e8c0072d210-must-gather-output\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.894313 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.894283    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:25:02.959424 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.959395    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbj62\" (UniqueName: \"kubernetes.io/projected/b24175b8-d015-439b-8c61-7e8c0072d210-kube-api-access-bbj62\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.959579 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.959453    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b24175b8-d015-439b-8c61-7e8c0072d210-must-gather-output\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.959763 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.959744    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/b24175b8-d015-439b-8c61-7e8c0072d210-must-gather-output\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.967512 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.967485    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbj62\" (UniqueName: \"kubernetes.io/projected/b24175b8-d015-439b-8c61-7e8c0072d210-kube-api-access-bbj62\") pod \"must-gather-d96xj\" (UID: \"b24175b8-d015-439b-8c61-7e8c0072d210\") " pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:02.981591 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:02.981570    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/must-gather-d96xj"
Sep 09 13:25:03.287917 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:03.287844    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/must-gather-d96xj"]
Sep 09 13:25:03.291215 ip-10-0-137-1 kubenswrapper[2580]: W0909 13:25:03.291189    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb24175b8_d015_439b_8c61_7e8c0072d210.slice/crio-accc07fb524b8faa0c7f2619fbca06b0530116d67227131f148305654e2917bb WatchSource:0}: Error finding container accc07fb524b8faa0c7f2619fbca06b0530116d67227131f148305654e2917bb: Status 404 returned error can't find the container with id accc07fb524b8faa0c7f2619fbca06b0530116d67227131f148305654e2917bb
Sep 09 13:25:03.436191 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:03.436153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/must-gather-d96xj" event={"ID":"b24175b8-d015-439b-8c61-7e8c0072d210","Type":"ContainerStarted","Data":"accc07fb524b8faa0c7f2619fbca06b0530116d67227131f148305654e2917bb"}
Sep 09 13:25:04.453276 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:04.453243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/must-gather-d96xj" event={"ID":"b24175b8-d015-439b-8c61-7e8c0072d210","Type":"ContainerStarted","Data":"8c69052704714ae78563e1fb636d7e17ff122833de96a6cb6c1940b9fd4c427e"}
Sep 09 13:25:04.792646 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:04.792605    2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log" limit=1024
Sep 09 13:25:05.457609 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:05.457568    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/must-gather-d96xj" event={"ID":"b24175b8-d015-439b-8c61-7e8c0072d210","Type":"ContainerStarted","Data":"c2f2439a4a33bcacc4dd9d7e9aab295e1a2930cb26a423559c87bcac1261fe9b"}
Sep 09 13:25:05.483700 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:05.483638    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sv79w/must-gather-d96xj" podStartSLOduration=2.544938161 podStartE2EDuration="3.483615092s" totalImagesPullingTime="938.676931ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 13:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:25:05.482146337 +0000 UTC m=+2203.517394245" watchObservedRunningTime="2026-09-09 13:25:05.483615092 +0000 UTC m=+2203.518863000"
Sep 09 13:25:05.985295 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:05.985260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-fj9l6_abd06375-8c1e-49ec-9a04-9782206cadbc/global-pull-secret-syncer/0.log"
Sep 09 13:25:06.119160 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:06.119075    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-2f5np_641d2aa8-d2be-4f45-abfe-5c18dde8f8fe/konnectivity-agent/0.log"
Sep 09 13:25:06.290333 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:06.290252    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-1.ec2.internal_270358c4069fad120920bbf1f2675138/haproxy/0.log"
Sep 09 13:25:10.490527 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:10.489959    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-56f8459dc5-m7glz_daa75492-94a7-46f2-b9e5-fa8df4e3c09d/authorino/0.log"
Sep 09 13:25:10.589924 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:10.589210    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-pmb8j_9e0d4df1-2852-4075-a8a8-0280475be7f3/manager/0.log"
Sep 09 13:25:10.631204 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:10.631089    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5tkgp_f5f610ea-dba0-4363-8f43-1f345023176a/manager/0.log"
Sep 09 13:25:10.708933 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:10.708895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-dcggn_e2fba6a9-8411-4803-8b9b-703c17f193dc/manager/0.log"
Sep 09 13:25:10.825737 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:10.825642    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-4v6zj_d60e23d2-c572-4c93-ab81-59f477b7e4b2/manager/0.log"
Sep 09 13:25:12.615952 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.615913    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6cdcc96965-9kn6t_7f392a16-e3e2-4bb1-b745-2575ec7debe7/cluster-monitoring-operator/0.log"
Sep 09 13:25:12.658045 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.658010    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-nnm6m_acab4b9d-26c3-466d-b619-72409878a2e5/kube-state-metrics/0.log"
Sep 09 13:25:12.684370 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.684305    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-nnm6m_acab4b9d-26c3-466d-b619-72409878a2e5/kube-rbac-proxy-main/0.log"
Sep 09 13:25:12.709436 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.709399    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-nnm6m_acab4b9d-26c3-466d-b619-72409878a2e5/kube-rbac-proxy-self/0.log"
Sep 09 13:25:12.784368 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.784337    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-z8l7h_176a869c-96ea-4c00-9758-3b5c06c8e4ef/monitoring-plugin/0.log"
Sep 09 13:25:12.831734 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.831693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64hhn_4619d408-59d1-4479-bdbc-569b9c104531/node-exporter/0.log"
Sep 09 13:25:12.862706 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.862672    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64hhn_4619d408-59d1-4479-bdbc-569b9c104531/kube-rbac-proxy/0.log"
Sep 09 13:25:12.904959 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:12.904933    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-64hhn_4619d408-59d1-4479-bdbc-569b9c104531/init-textfile/0.log"
Sep 09 13:25:15.177180 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.177137    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-fj4kt_cad0a6c1-e143-45f1-a7f3-08826bc0c048/networking-console-plugin/0.log"
Sep 09 13:25:15.609463 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.609367    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"]
Sep 09 13:25:15.627887 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.627301    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.633448 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.633419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"]
Sep 09 13:25:15.689749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.688922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxs9j\" (UniqueName: \"kubernetes.io/projected/b7018eec-d15d-4104-bdbd-732908a4c32c-kube-api-access-sxs9j\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.689749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.688999    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-podres\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.689749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.689041    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-proc\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.689749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.689111    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-sys\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.689749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.689187    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-lib-modules\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790524    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxs9j\" (UniqueName: \"kubernetes.io/projected/b7018eec-d15d-4104-bdbd-732908a4c32c-kube-api-access-sxs9j\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790568 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790571    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-podres\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790792 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790688    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-proc\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790792 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-podres\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790792 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-proc\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790792 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790766    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-sys\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790817    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-lib-modules\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-sys\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.790985 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.790929    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b7018eec-d15d-4104-bdbd-732908a4c32c-lib-modules\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.806639 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.806610    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxs9j\" (UniqueName: \"kubernetes.io/projected/b7018eec-d15d-4104-bdbd-732908a4c32c-kube-api-access-sxs9j\") pod \"perf-node-gather-daemonset-hjbcb\" (UID: \"b7018eec-d15d-4104-bdbd-732908a4c32c\") " pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:15.886557 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.886458    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/2.log"
Sep 09 13:25:15.895308 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.895280    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-cg7vj_42bd9e84-8866-4e8c-a958-a96cd635510b/console-operator/3.log"
Sep 09 13:25:15.943749 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:15.943718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:16.259802 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.259770    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"]
Sep 09 13:25:16.490255 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.490225    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64d54f989d-4wmhs_825510a8-2d4b-4adc-a7fd-2f5f4c501683/console/0.log"
Sep 09 13:25:16.570406 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.570305    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb" event={"ID":"b7018eec-d15d-4104-bdbd-732908a4c32c","Type":"ContainerStarted","Data":"d6f984c8ae25c7544693ef661d0ff133d7c582a0b75c23f462cf32a957c0b5c8"}
Sep 09 13:25:16.570406 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.570374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb" event={"ID":"b7018eec-d15d-4104-bdbd-732908a4c32c","Type":"ContainerStarted","Data":"a020f2bcc18fcf38b49838f371f87ee66e74f303c27ac232dc748a84972d5511"}
Sep 09 13:25:16.571169 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.571141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:16.594187 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:16.594122    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb" podStartSLOduration=1.594103144 podStartE2EDuration="1.594103144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 13:25:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 13:25:16.590316431 +0000 UTC m=+2214.625564356" watchObservedRunningTime="2026-09-09 13:25:16.594103144 +0000 UTC m=+2214.629351052"
Sep 09 13:25:17.249885 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:17.249847    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-695c895bd6-z9s54_6607ac44-3eff-4dbc-982f-8c3902f790ee/volume-data-source-validator/0.log"
Sep 09 13:25:18.519990 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:18.519959    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zr2l7_45fa2142-a4b5-405b-80a4-27438f6e9080/dns/0.log"
Sep 09 13:25:18.543244 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:18.543218    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-zr2l7_45fa2142-a4b5-405b-80a4-27438f6e9080/kube-rbac-proxy/0.log"
Sep 09 13:25:18.572366 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:18.572332    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-8ngjq_853f7946-ed3b-44b7-a2d8-fc6e57cf18b0/dns-node-resolver/0.log"
Sep 09 13:25:19.242802 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:19.242775    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-rc72g_29c984e6-6c2f-4784-bb63-3b13e74b7af6/node-ca/0.log"
Sep 09 13:25:20.720234 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:20.720200    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-768599446d-wl78j_a6934b57-ea63-4e20-8245-2a833e114f76/kube-auth-proxy/0.log"
Sep 09 13:25:20.891672 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:20.891640    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:25:20.921454 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:20.921421    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b548b666-qdt2z_e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3/router/0.log"
Sep 09 13:25:21.833445 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:21.833416    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2l9tv_6153e58f-438f-44f4-a984-421cae427aea/serve-healthcheck-canary/0.log"
Sep 09 13:25:22.398973 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:22.398943    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rp5wd_66e4f7f3-6970-4189-b923-865fbac766d2/insights-operator/1.log"
Sep 09 13:25:22.401219 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:22.401194    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-rp5wd_66e4f7f3-6970-4189-b923-865fbac766d2/insights-operator/0.log"
Sep 09 13:25:22.613606 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:22.613580    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hxhf7_82213e13-c14f-4841-9ae5-7fbbe5f150be/kube-rbac-proxy/0.log"
Sep 09 13:25:22.641758 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:22.641729    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hxhf7_82213e13-c14f-4841-9ae5-7fbbe5f150be/exporter/0.log"
Sep 09 13:25:22.667814 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:22.667738    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-hxhf7_82213e13-c14f-4841-9ae5-7fbbe5f150be/extractor/0.log"
Sep 09 13:25:23.094211 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:23.094130    2580 scope.go:117] "RemoveContainer" containerID="87133451e2c8753ce555b7a3c6c13e84e7a285b03e5620d6b8963e37317fd7f1"
Sep 09 13:25:23.105708 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:23.105687    2580 scope.go:117] "RemoveContainer" containerID="657fbce0a0b1941d21950d8c8c3ec98b9d4e74bfbb9ad7ce08b3f51a57f4469b"
Sep 09 13:25:23.600756 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:23.600726    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-sv79w/perf-node-gather-daemonset-hjbcb"
Sep 09 13:25:25.375271 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:25.375176    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-7gzcx_0d60c981-2889-4148-a482-9588c452044e/manager/0.log"
Sep 09 13:25:25.721878 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:25.721846    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-9ppks_8bf8f3b2-e153-424b-9805-35ab0c027117/manager/0.log"
Sep 09 13:25:25.793778 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:25.793744    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-885499b4f-drpss_1da7c2b9-c716-417e-9eb2-639f89a2948f/manager/0.log"
Sep 09 13:25:26.164281 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:26.164245    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-dxwnv_ec32ce95-8011-4a26-b5c1-3b9e3138eeb5/manager/0.log"
Sep 09 13:25:26.185250 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:26.185224    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-647774d94-dxwnv_ec32ce95-8011-4a26-b5c1-3b9e3138eeb5/copy-manifests/0.log"
Sep 09 13:25:26.587740 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:26.587656    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-768599446d-wl78j_a6934b57-ea63-4e20-8245-2a833e114f76/kube-auth-proxy/0.log"
Sep 09 13:25:26.719539 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:26.719484    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:25:26.741101 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:26.741072    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b548b666-qdt2z_e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3/router/0.log"
Sep 09 13:25:27.715471 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:27.715433    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-5zkqj_207dfaa1-41ee-4e9a-97c8-7c4ef0b41c15/openshift-lws-operator/0.log"
Sep 09 13:25:28.687521 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:28.687471    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-768599446d-wl78j_a6934b57-ea63-4e20-8245-2a833e114f76/kube-auth-proxy/0.log"
Sep 09 13:25:28.793931 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:28.793895    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-wq5fl_adca4535-86ea-4c97-8b3e-1e1636b4ab0d/payload-processing/0.log"
Sep 09 13:25:28.816536 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:28.816478    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-b548b666-qdt2z_e3a5edb6-f4ac-4532-8fbe-ac6dc8fbffd3/router/0.log"
Sep 09 13:25:32.599204 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:32.599172    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sp942_b65e71dc-6b03-438b-aa9d-20dd02df2f1b/migrator/0.log"
Sep 09 13:25:32.623083 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:32.623014    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-sp942_b65e71dc-6b03-438b-aa9d-20dd02df2f1b/graceful-termination/0.log"
Sep 09 13:25:33.068133 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:33.068078    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5gsr9_5613ea48-561a-49c6-84da-94fa313b1668/kube-storage-version-migrator-operator/1.log"
Sep 09 13:25:33.070371 ip-10-0-137-1 kubenswrapper[2580]: I0909 13:25:33.070338    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-5gsr9_5613ea48-561a-49c6-84da-94fa313b1668/kube-storage-version-migrator-operator/0.log"