Sep 08 13:26:28.777494 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 08 13:26:28.777503 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 08 13:26:28.777510 ip-10-0-140-23 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 08 13:26:28.777662 ip-10-0-140-23 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 08 13:26:38.896346 ip-10-0-140-23 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 08 13:26:38.896365 ip-10-0-140-23 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e48e5f4bfbda4aadb3753c2030815632 --
Sep 08 13:28:33.675863 ip-10-0-140-23 systemd[1]: Starting Kubernetes Kubelet...
Sep 08 13:28:34.038272 ip-10-0-140-23 kubenswrapper[2583]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:34.038272 ip-10-0-140-23 kubenswrapper[2583]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 08 13:28:34.038272 ip-10-0-140-23 kubenswrapper[2583]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:34.038925 ip-10-0-140-23 kubenswrapper[2583]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 08 13:28:34.038925 ip-10-0-140-23 kubenswrapper[2583]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 08 13:28:34.040508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.040421    2583 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 08 13:28:34.044012 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.043997    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044013    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044017    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044021    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044024    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044027    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044031    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044035    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044038    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044041    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044044    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044046    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044049    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044052    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.044050 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044056    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044059    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044062    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044065    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044067    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044075    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044078    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044081    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044083    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044086    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044088    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044091    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044093    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044096    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044098    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044101    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044103    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044105    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044108    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.044385 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044110    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044113    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044115    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044118    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044122    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044126    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044128    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044131    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044135    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044138    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044141    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044144    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044146    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044149    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044152    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044155    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044157    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044160    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044162    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.044853 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044165    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044167    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044170    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044172    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044174    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044177    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044179    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044182    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044184    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044187    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044190    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044192    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044195    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044197    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044199    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044203    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044206    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044208    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044210    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044213    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.045326 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044216    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044218    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044221    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044224    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044226    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044229    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044233    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044236    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044239    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044242    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044245    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044248    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044250    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044253    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044256    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044258    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044261    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044264    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044267    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044270    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.045817 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044272    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044275    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044279    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044282    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044284    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044287    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.044289    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045800    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045809    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045812    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045815    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045818    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045821    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045824    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045827    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045830    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045833    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045836    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045839    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045841    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.046300 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045843    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045846    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045848    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045851    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045854    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045856    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045858    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045861    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045864    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045866    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045869    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045871    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045874    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045877    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045880    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045883    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045885    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045888    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045892    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.046837 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045895    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045898    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045901    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045903    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045906    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045909    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045912    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045914    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045917    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045919    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045922    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045926    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045929    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045932    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045936    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045939    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045942    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045944    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045946    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045949    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.047299 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045951    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045954    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045956    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045959    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045961    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045963    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045966    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045969    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045972    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045974    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045977    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045979    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045982    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045984    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045987    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045990    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045992    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045994    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045997    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.045999    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.047813 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046002    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046004    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046006    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046009    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046012    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046014    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046016    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046019    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046021    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046024    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046026    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046030    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046032    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046034    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046037    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046039    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046042    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046044    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046047    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046051    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.048323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046053    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046056    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046058    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046061    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046063    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046067    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046070    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046425    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046432    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046436    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046440    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046443    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046447    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046451    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046456    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046460    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046464    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046466    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046469    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046471    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.048818 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046474    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046477    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046479    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046482    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046484    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046486    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046489    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046491    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046495    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046499    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046504    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046508    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046511    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046514    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046517    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046520    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046522    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046526    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046529    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.049304 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046531    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046534    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046536    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046539    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046541    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046544    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046560    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046563    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046565    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046569    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046571    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046574    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046577    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046579    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046582    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046584    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046587    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046589    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046592    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046594    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.049815 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046598    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046601    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046603    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046605    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046608    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046611    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046613    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046616    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046619    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046621    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046624    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046626    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046629    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046631    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046634    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046636    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046639    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046641    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046644    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046646    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.050400 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046648    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046651    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046654    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046656    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046659    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046661    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046663    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046666    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046669    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046671    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046673    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046676    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046678    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046681    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046683    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046685    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046689    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046691    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046694    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046696    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.050900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046699    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046702    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046706    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046708    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046711    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046713    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.046715    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046799    2583 flags.go:64] FLAG: --address="0.0.0.0"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046833    2583 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046840    2583 flags.go:64] FLAG: --anonymous-auth="true"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046846    2583 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046850    2583 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046853    2583 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046857    2583 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046861    2583 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046865    2583 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046868    2583 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046872    2583 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046875    2583 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046878    2583 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046881    2583 flags.go:64] FLAG: --cgroup-root=""
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046884    2583 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 08 13:28:34.051389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046887    2583 flags.go:64] FLAG: --client-ca-file=""
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046889    2583 flags.go:64] FLAG: --cloud-provider="external"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046892    2583 flags.go:64] FLAG: --cluster-dns="[]"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046896    2583 flags.go:64] FLAG: --cluster-domain=""
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046899    2583 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046902    2583 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046906    2583 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046909    2583 flags.go:64] FLAG: --container-log-max-files="5"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046913    2583 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046915    2583 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046918    2583 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046921    2583 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046924    2583 flags.go:64] FLAG: --contention-profiling="false"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046927    2583 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046930    2583 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046933    2583 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046935    2583 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046939    2583 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046942    2583 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046945    2583 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046948    2583 flags.go:64] FLAG: --enable-load-reader="false"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046950    2583 flags.go:64] FLAG: --enable-server="true"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046953    2583 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046958    2583 flags.go:64] FLAG: --event-burst="100"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046961    2583 flags.go:64] FLAG: --event-qps="50"
Sep 08 13:28:34.051980 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046964    2583 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046968    2583 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046970    2583 flags.go:64] FLAG: --eviction-hard=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046974    2583 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046979    2583 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046982    2583 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046985    2583 flags.go:64] FLAG: --eviction-soft=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046988    2583 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046990    2583 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.046993    2583 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047009    2583 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047013    2583 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047017    2583 flags.go:64] FLAG: --fail-swap-on="true"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047020    2583 flags.go:64] FLAG: --feature-gates=""
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047024    2583 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047027    2583 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047030    2583 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047033    2583 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047037    2583 flags.go:64] FLAG: --healthz-port="10248"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047039    2583 flags.go:64] FLAG: --help="false"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047042    2583 flags.go:64] FLAG: --hostname-override="ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047045    2583 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047048    2583 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 08 13:28:34.052595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047051    2583 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047054    2583 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047057    2583 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047059    2583 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047062    2583 flags.go:64] FLAG: --image-service-endpoint=""
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047065    2583 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047068    2583 flags.go:64] FLAG: --kube-api-burst="100"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047070    2583 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047073    2583 flags.go:64] FLAG: --kube-api-qps="50"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047076    2583 flags.go:64] FLAG: --kube-reserved=""
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047079    2583 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047088    2583 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047091    2583 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047096    2583 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047099    2583 flags.go:64] FLAG: --lock-file=""
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047101    2583 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047104    2583 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047107    2583 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047116    2583 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047119    2583 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047121    2583 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047124    2583 flags.go:64] FLAG: --logging-format="text"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047128    2583 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047131    2583 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 08 13:28:34.053174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047134    2583 flags.go:64] FLAG: --manifest-url=""
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047137    2583 flags.go:64] FLAG: --manifest-url-header=""
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047141    2583 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047144    2583 flags.go:64] FLAG: --max-open-files="1000000"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047148    2583 flags.go:64] FLAG: --max-pods="110"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047151    2583 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047153    2583 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047156    2583 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047159    2583 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047162    2583 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047165    2583 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047168    2583 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047175    2583 flags.go:64] FLAG: --node-status-max-images="50"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047178    2583 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047181    2583 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047184    2583 flags.go:64] FLAG: --pod-cidr=""
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047187    2583 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047192    2583 flags.go:64] FLAG: --pod-manifest-path=""
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047194    2583 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047197    2583 flags.go:64] FLAG: --pods-per-core="0"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047201    2583 flags.go:64] FLAG: --port="10250"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047206    2583 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047209    2583 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-062ce07c734edcf2d"
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047212    2583 flags.go:64] FLAG: --qos-reserved=""
Sep 08 13:28:34.053774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047215    2583 flags.go:64] FLAG: --read-only-port="10255"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047218    2583 flags.go:64] FLAG: --register-node="true"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047221    2583 flags.go:64] FLAG: --register-with-taints=""
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047224    2583 flags.go:64] FLAG: --registry-burst="10"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047227    2583 flags.go:64] FLAG: --registry-qps="5"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047230    2583 flags.go:64] FLAG: --reserved-cpus=""
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047232    2583 flags.go:64] FLAG: --reserved-memory=""
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047236    2583 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047239    2583 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047242    2583 flags.go:64] FLAG: --rotate-certificates="false"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047244    2583 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047247    2583 flags.go:64] FLAG: --runonce="false"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047250    2583 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047252    2583 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047255    2583 flags.go:64] FLAG: --seccomp-default="false"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047258    2583 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047260    2583 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047263    2583 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047266    2583 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047268    2583 flags.go:64] FLAG: --storage-driver-password="root"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047271    2583 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047274    2583 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047277    2583 flags.go:64] FLAG: --storage-driver-user="root"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047279    2583 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047282    2583 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 08 13:28:34.054351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047285    2583 flags.go:64] FLAG: --system-cgroups=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047287    2583 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047293    2583 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047295    2583 flags.go:64] FLAG: --tls-cert-file=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047300    2583 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047304    2583 flags.go:64] FLAG: --tls-min-version=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047307    2583 flags.go:64] FLAG: --tls-private-key-file=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047310    2583 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047312    2583 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047315    2583 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047318    2583 flags.go:64] FLAG: --v="2"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047322    2583 flags.go:64] FLAG: --version="false"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047326    2583 flags.go:64] FLAG: --vmodule=""
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047330    2583 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.047334    2583 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047447    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047451    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047454    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047457    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047460    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047462    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047465    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047468    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.054979 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047470    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047473    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047475    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047478    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047480    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047482    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047485    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047487    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047490    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047492    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047495    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047497    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047500    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047504    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047507    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047509    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047512    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047514    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047517    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047519    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.055589 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047522    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047524    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047527    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047531    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047534    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047536    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047539    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047541    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047544    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047564    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047568    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047572    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047575    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047577    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047580    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047582    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047585    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047587    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047590    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.056101 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047592    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047595    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047597    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047600    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047603    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047606    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047611    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047613    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047616    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047618    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047621    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047624    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047627    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047629    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047632    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047634    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047638    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047640    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047643    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047645    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.056571 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047649    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047653    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047655    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047658    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047661    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047664    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047666    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047669    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047671    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047674    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047676    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047678    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047681    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047684    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047688    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047691    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047695    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047698    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047703    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.057069 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047706    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047710    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047714    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047717    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047720    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047723    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047726    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047729    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047731    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047735    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047738    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047741    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.047743    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.048328    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.055477    2583 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 08 13:28:34.057535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.055492    2583 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.056288    2583 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.056295    2583 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056352    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056357    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056361    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056363    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056366    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056369    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056372    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056375    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056378    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056380    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056383    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056386    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056389    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056393    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056397    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056400    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.057932 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056403    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056406    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056409    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056411    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056415    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056417    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056420    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056423    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056426    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056428    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056430    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056433    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056435    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056438    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056441    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056443    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056446    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056449    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056452    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056454    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.058394 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056456    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056459    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056461    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056464    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056467    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056470    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056472    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056475    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056477    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056480    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056482    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056484    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056487    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056490    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056494    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056496    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056499    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056502    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056505    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056507    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.058931 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056509    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056512    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056514    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056516    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056519    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056522    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056525    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056527    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056530    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056532    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056536    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056539    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056542    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056545    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056563    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056567    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056570    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056572    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056576    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056578    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.059429 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056581    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056584    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056586    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056589    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056592    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056594    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056597    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056600    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056603    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056606    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056608    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056610    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056613    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056615    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056618    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056620    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056623    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056625    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056627    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056630    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.060017 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056632    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056634    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056638    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.056643    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056760    2583 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056765    2583 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056769    2583 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056773    2583 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056776    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056779    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056782    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056785    2583 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056788    2583 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056791    2583 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 08 13:28:34.060497 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056795    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056798    2583 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056800    2583 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056803    2583 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056805    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056808    2583 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056811    2583 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056814    2583 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056816    2583 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056819    2583 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056821    2583 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056824    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056827    2583 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056829    2583 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056832    2583 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056835    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056837    2583 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056840    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056843    2583 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056845    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 08 13:28:34.060858 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056848    2583 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056850    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056853    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056855    2583 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056857    2583 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056860    2583 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056862    2583 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056865    2583 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056867    2583 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056870    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056872    2583 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056874    2583 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056877    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056880    2583 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056882    2583 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056884    2583 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056887    2583 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056889    2583 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056892    2583 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056895    2583 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 08 13:28:34.061354 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056898    2583 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056900    2583 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056903    2583 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056905    2583 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056907    2583 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056910    2583 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056912    2583 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056915    2583 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056917    2583 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056921    2583 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056924    2583 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056927    2583 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056930    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056933    2583 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056936    2583 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056938    2583 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056941    2583 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056943    2583 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056945    2583 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056948    2583 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 08 13:28:34.061852 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056950    2583 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056953    2583 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056955    2583 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056957    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056960    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056962    2583 feature_gate.go:328] unrecognized feature gate: Example2
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056965    2583 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056967    2583 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056970    2583 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056972    2583 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056975    2583 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056978    2583 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056980    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056983    2583 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056985    2583 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056988    2583 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056990    2583 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056992    2583 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056995    2583 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.056997    2583 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 08 13:28:34.062328 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057000    2583 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057002    2583 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057004    2583 feature_gate.go:328] unrecognized feature gate: Example
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057007    2583 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057009    2583 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057012    2583 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057014    2583 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057016    2583 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:34.057019    2583 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.057024    2583 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.057127    2583 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.059708    2583 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.060615    2583 server.go:1019] "Starting client certificate rotation"
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.060722    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:28:34.062838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.060764    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 08 13:28:34.063648 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.063635    2583 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:28:34.067295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.067268    2583 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 08 13:28:34.078201 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.078182    2583 log.go:25] "Validated CRI v1 runtime API"
Sep 08 13:28:34.081620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.081600    2583 log.go:25] "Validated CRI v1 image API"
Sep 08 13:28:34.082528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.082513    2583 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 08 13:28:34.084881 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.084854    2583 fs.go:135] Filesystem UUIDs: map[26c83b4c-ac91-4f57-9469-0f28f7ac41bf:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 865801db-78d6-442d-acfb-2ca47126e7ff:/dev/nvme0n1p4]
Sep 08 13:28:34.084957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.084876    2583 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 08 13:28:34.090361 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.090235    2583 manager.go:216] Machine: {Timestamp:2026-09-08 13:28:34.088577032 +0000 UTC m=+0.306493359 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102444 MemoryCapacity:32812154880 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20c20376d2a8c83244994b4aa4de14 SystemUUID:ec20c203-76d2-a8c8-3244-994b4aa4de14 BootID:e48e5f4b-fbda-4aad-b375-3c2030815632 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406077440 Type:vfs Inodes:4005390 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562430976 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406077440 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:9d:11:11:3a:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:9d:11:11:3a:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:02:8f:80:96:ba:1a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812154880 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 08 13:28:34.090361 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.090349    2583 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 08 13:28:34.090526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.090465    2583 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 08 13:28:34.093325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.093292    2583 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 08 13:28:34.093506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.093330    2583 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-23.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 08 13:28:34.094108 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.094092    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:28:34.094597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.094582    2583 topology_manager.go:138] "Creating topology manager with none policy"
Sep 08 13:28:34.094664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.094604    2583 container_manager_linux.go:309] "Creating device plugin manager"
Sep 08 13:28:34.094664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.094615    2583 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:28:34.095304 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.095292    2583 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 08 13:28:34.096502 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.096489    2583 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 08 13:28:34.100606 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100593    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:28:34.100741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100730    2583 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 08 13:28:34.100809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100768    2583 kubelet.go:486] "Attempting to sync node with API server"
Sep 08 13:28:34.100809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100783    2583 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 08 13:28:34.100809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100804    2583 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 08 13:28:34.100934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100829    2583 kubelet.go:398] "Adding apiserver pod source"
Sep 08 13:28:34.100934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.100852    2583 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 08 13:28:34.101230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.101215    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:28:34.101291 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.101239    2583 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 08 13:28:34.103367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.103346    2583 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 08 13:28:34.106635 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106619    2583 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 08 13:28:34.106706 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106651    2583 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 08 13:28:34.106807 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106797    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 08 13:28:34.106840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106812    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 08 13:28:34.106840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106819    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 08 13:28:34.106840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106827    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 08 13:28:34.106840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106838    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 08 13:28:34.106978 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.106846    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 08 13:28:34.107227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107215    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 08 13:28:34.107262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107231    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 08 13:28:34.107262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107242    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 08 13:28:34.107262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107251    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 08 13:28:34.107338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107264    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 08 13:28:34.107338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.107275    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 08 13:28:34.108013 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.108004    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 08 13:28:34.108046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.108014    2583 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 08 13:28:34.110943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.110923    2583 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-23.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 08 13:28:34.111358 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.111339    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 08 13:28:34.111390 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.111340    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 08 13:28:34.112100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.112088    2583 server.go:1268] "Started kubelet"
Sep 08 13:28:34.112204 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.112162    2583 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 08 13:28:34.112322 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.112162    2583 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 08 13:28:34.112382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.112342    2583 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 08 13:28:34.112983 ip-10-0-140-23 systemd[1]: Started Kubernetes Kubelet.
Sep 08 13:28:34.114435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.114301    2583 server.go:310] "Adding debug handlers to kubelet server"
Sep 08 13:28:34.116253 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.116239    2583 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 08 13:28:34.117164 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.117143    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg4x8"
Sep 08 13:28:34.118423 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.118403    2583 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 08 13:28:34.119582 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.118730    2583 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-23.ec2.internal.18d35b87678a7f62  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-23.ec2.internal,UID:ip-10-0-140-23.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-23.ec2.internal,},FirstTimestamp:2026-09-08 13:28:34.112061282 +0000 UTC m=+0.329977597,LastTimestamp:2026-09-08 13:28:34.112061282 +0000 UTC m=+0.329977597,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-23.ec2.internal,}"
Sep 08 13:28:34.120229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.120214    2583 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 08 13:28:34.120771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.120757    2583 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 08 13:28:34.120880 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.120864    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.121401 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.121387    2583 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 08 13:28:34.121401 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.121389    2583 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 08 13:28:34.121499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.121406    2583 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 08 13:28:34.121545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.121513    2583 reconstruct.go:97] "Volume reconstruction finished"
Sep 08 13:28:34.121545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.121523    2583 reconciler.go:29] "Reconciler: start to sync state"
Sep 08 13:28:34.122174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122156    2583 factory.go:55] Registering systemd factory
Sep 08 13:28:34.122248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122178    2583 factory.go:223] Registration of the systemd container factory successfully
Sep 08 13:28:34.122363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122332    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zg4x8"
Sep 08 13:28:34.122461 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122395    2583 factory.go:153] Registering CRI-O factory
Sep 08 13:28:34.122461 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122412    2583 factory.go:223] Registration of the crio container factory successfully
Sep 08 13:28:34.122612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122471    2583 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 08 13:28:34.122612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122487    2583 factory.go:103] Registering Raw factory
Sep 08 13:28:34.122612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122498    2583 manager.go:1196] Started watching for new ooms in manager
Sep 08 13:28:34.122796 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.122784    2583 manager.go:349] Starting recovery of all containers
Sep 08 13:28:34.124304 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.124276    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 08 13:28:34.124401 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.124364    2583 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-23.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 08 13:28:34.132342 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.132319    2583 manager.go:354] Recovery completed
Sep 08 13:28:34.137368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.137355    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.139275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139260    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.139333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139294    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.139333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139309    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.139932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139918    2583 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 08 13:28:34.139932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139930    2583 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 08 13:28:34.140036 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139945    2583 state_mem.go:36] "Initialized new in-memory state store"
Sep 08 13:28:34.140036 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.139919    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.140677 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.140664    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.140730 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.140689    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.140730 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.140698    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.141834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.141823    2583 policy_none.go:49] "None policy: Start"
Sep 08 13:28:34.141876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.141838    2583 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 08 13:28:34.141876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.141848    2583 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 08 13:28:34.143707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.143697    2583 policy_none.go:47] "Start"
Sep 08 13:28:34.177159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177145    2583 manager.go:337] "Starting Device Plugin manager"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.177187    2583 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177198    2583 server.go:87] "Starting device plugin registration server"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177535    2583 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177612    2583 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177735    2583 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177844    2583 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.177853    2583 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.178368    2583 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 08 13:28:34.205046 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.178421    2583 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.222941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.222914    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 08 13:28:34.224182 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.224166    2583 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 08 13:28:34.224263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.224219    2583 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 08 13:28:34.224263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.224248    2583 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 08 13:28:34.224350 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.224294    2583 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 08 13:28:34.226833 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.226815    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:34.277753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.277710    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.278669 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.278650    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.278763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.278687    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.278763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.278701    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.278763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.278725    2583 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.286783 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.286767    2583 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.286835 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.286793    2583 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-23.ec2.internal\": node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.300513 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.300422    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.325195 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.325166    2583 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"]
Sep 08 13:28:34.325325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.325247    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.326130 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.326114    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.326213 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.326149    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.326213 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.326165    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.328382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.328367    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.328491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.328477    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.328536 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.328509    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.329045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329029    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.329114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329056    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.329114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329070    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.329114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329032    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.329114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329099    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.329114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.329114    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.331190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.331176    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.331250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.331201    2583 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 08 13:28:34.331823 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.331807    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientMemory"
Sep 08 13:28:34.331945 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.331834    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasNoDiskPressure"
Sep 08 13:28:34.331945 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.331843    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeHasSufficientPID"
Sep 08 13:28:34.360428 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.360401    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-23.ec2.internal\" not found" node="ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.364759 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.364741    2583 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-23.ec2.internal\" not found" node="ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.401364 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.401338    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.422654 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.422627    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.422743 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.422661    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.422743 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.422688    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8ce096f9d3777e9f36fc81bb4086cc7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"b8ce096f9d3777e9f36fc81bb4086cc7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.501432 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.501395    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.523846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523815    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.523846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523839    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.523977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523875    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.523977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8ce096f9d3777e9f36fc81bb4086cc7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"b8ce096f9d3777e9f36fc81bb4086cc7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.523977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523924    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/b8ce096f9d3777e9f36fc81bb4086cc7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-23.ec2.internal\" (UID: \"b8ce096f9d3777e9f36fc81bb4086cc7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.523977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.523949    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8cd9ea59b49ae21cabe35a064ca81a74-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal\" (UID: \"8cd9ea59b49ae21cabe35a064ca81a74\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.602161 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.602080    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.663658 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.663634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.667259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.667238    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:34.703016 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.702983    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.803627 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.803579    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.904127 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:34.904047    2583 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-23.ec2.internal\" not found"
Sep 08 13:28:34.952530 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:34.952503    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:35.021582 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.021540    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:35.031025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.031007    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:28:35.032773 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.032761    2583 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal"
Sep 08 13:28:35.042233 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.042218    2583 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 08 13:28:35.060382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.060362    2583 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 08 13:28:35.060500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.060482    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 13:28:35.060575 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.060526    2583 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 08 13:28:35.101733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.101697    2583 apiserver.go:52] "Watching apiserver"
Sep 08 13:28:35.118684 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.118651    2583 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 08 13:28:35.118994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.118973    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-ldrh2","kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal","openshift-multus/multus-94lfj","openshift-multus/network-metrics-daemon-msdpw","openshift-network-operator/iptables-alerter-s2hwk","openshift-ovn-kubernetes/ovnkube-node-9fth2","kube-system/global-pull-secret-syncer-ngbdv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2","openshift-cluster-node-tuning-operator/tuned-9ds6k","openshift-dns/node-resolver-ljkxh","openshift-image-registry/node-ca-lt4dq","openshift-multus/multus-additional-cni-plugins-chtg8","openshift-network-diagnostics/network-check-target-zjktc"]
Sep 08 13:28:35.120452 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.120437    2583 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 08 13:28:35.121476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.121463    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.123568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.123538    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.123974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.123673    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 08 13:28:35.124113 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.124094    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 08 13:28:35.125042 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.124997    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-07 13:23:34 +0000 UTC" deadline="2028-06-24 19:49:21.931609066 +0000 UTC"
Sep 08 13:28:35.125042 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.125039    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15726h20m46.806573513s"
Sep 08 13:28:35.125444 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.125423    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.125735    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.125770    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.125967    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126013    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126104    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-8mjtt\""
Sep 08 13:28:35.126160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126117    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9qg79\""
Sep 08 13:28:35.126450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126363    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 08 13:28:35.126844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126676    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.126951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126875    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.126951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126886    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:35.127070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.126992    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127368    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127401    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.127400    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-system-cni-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bpb58\" (UniqueName: \"kubernetes.io/projected/91344415-9650-42ef-a460-f8a3029a9100-kube-api-access-bpb58\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-kubelet\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127048    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127544    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-netd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-config\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e22f156-742a-4317-aafc-f354d6f9a56c-ovn-node-metrics-cert\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127617    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-slash\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127632    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-bin\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127689    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-env-overrides\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127735    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-var-lib-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127785    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-etc-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-log-socket\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127847    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vqcn\" (UniqueName: \"kubernetes.io/projected/5e22f156-742a-4317-aafc-f354d6f9a56c-kube-api-access-6vqcn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127876    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-systemd-units\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-netns\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-cnibin\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.127969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127968    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-systemd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.128399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.127988    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.128399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.128027    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-ovn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.128399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.128057    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-node-log\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.128399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.128083    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-script-lib\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.129290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.129273    2583 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 08 13:28:35.129526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.129512    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.131365 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.131352    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 08 13:28:35.131433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.131419    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dkk6g\""
Sep 08 13:28:35.131757 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.131739    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.131835 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.131811    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:35.134099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.134086    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.136185 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.136169    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.136343 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.136319    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-bfk8r\""
Sep 08 13:28:35.136422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.136375    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.136422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.136321    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 08 13:28:35.136850 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.136836    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.139086 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.139065    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-zzt9g\""
Sep 08 13:28:35.139158 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.139085    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.139158 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.139092    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 08 13:28:35.139158 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.139151    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.139251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.139085    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.141301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.141279    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 08 13:28:35.141366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.141299    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-gqcmx\""
Sep 08 13:28:35.141417 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.141361    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.141417 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.141302    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 08 13:28:35.141493 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.141421    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:35.143702 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.143686    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.145682 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.145667    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.145859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.145840    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.145977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.145959    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.146043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.146027    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pqgsm\""
Sep 08 13:28:35.148232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.148215    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.148326 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.148219    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-22pgs\""
Sep 08 13:28:35.148326 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.148221    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.148482 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.148461    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.150264 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.150250    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 08 13:28:35.150330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.150319    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 08 13:28:35.150473 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.150458    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 08 13:28:35.150679 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.150666    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5mhlk\""
Sep 08 13:28:35.154585 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.154532    2583 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5bzhz"
Sep 08 13:28:35.161591 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.161578    2583 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5bzhz"
Sep 08 13:28:35.223119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.223092    2583 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 08 13:28:35.229118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229099    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-conf-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229124    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-run\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.229222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-cni-binary-copy\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229170    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-netns\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229215    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-conf\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229244    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229259    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-kubelet\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229284    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-systemd-units\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-cnibin\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229316    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-socket-dir-parent\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229337    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-hostroot\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229355    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-cnibin\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229358    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229361    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-systemd-units\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229386    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-systemd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229417    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229448    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-systemd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229476    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-node-log\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229494    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229517    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9b46\" (UniqueName: \"kubernetes.io/projected/b10de668-df4d-40b5-9e19-255843710763-kube-api-access-r9b46\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229539    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-node-log\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229543    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13910b21-8b7c-46fa-b444-194c280bd51c-konnectivity-ca\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229601    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zspv\" (UniqueName: \"kubernetes.io/projected/60a8bc96-a399-4cd8-9ebf-a0499e48b232-kube-api-access-6zspv\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229615    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-daemon-config\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229636    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bpb58\" (UniqueName: \"kubernetes.io/projected/91344415-9650-42ef-a460-f8a3029a9100-kube-api-access-bpb58\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229657    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e942d48-6e53-498f-8d2b-b9f88ca36d83-hosts-file\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60a8bc96-a399-4cd8-9ebf-a0499e48b232-host\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229684    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-system-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229697    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-etc-kubernetes\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.229724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229718    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-sys\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229739    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-kubelet\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229754    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-config\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e22f156-742a-4317-aafc-f354d6f9a56c-ovn-node-metrics-cert\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229782    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-k8s-cni-cncf-io\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-bin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-var-lib-kubelet\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229809    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-kubelet\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-slash\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229871    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-bin\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229900    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-bin\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229907    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-env-overrides\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-slash\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229956    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.229976    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-kubelet-config\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230065    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-dbus\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acf595f4-708b-416b-9dc3-166a6166a23e-host-slash\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.230378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230113    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-var-lib-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230136    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230158    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-var-lib-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230162    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-etc-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230194    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-etc-openvswitch\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-log-socket\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230229    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-tuning-conf-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230234    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230224    2583 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47mxr\" (UniqueName: \"kubernetes.io/projected/22924846-24ee-409a-a0fe-2acdc375fa2b-kube-api-access-47mxr\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230306    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-tmp\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230310    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-log-socket\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230334    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vqcn\" (UniqueName: \"kubernetes.io/projected/5e22f156-742a-4317-aafc-f354d6f9a56c-kube-api-access-6vqcn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230360    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/acf595f4-708b-416b-9dc3-166a6166a23e-iptables-alerter-script\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99d4v\" (UniqueName: \"kubernetes.io/projected/acf595f4-708b-416b-9dc3-166a6166a23e-kube-api-access-99d4v\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230408    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-netns\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230437    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e942d48-6e53-498f-8d2b-b9f88ca36d83-tmp-dir\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.231257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230450    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-config\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230433    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-env-overrides\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230484    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-netns\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230499    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230510    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-device-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-etc-selinux\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230587    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-sys-fs\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-ovn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-script-lib\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230658    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-run-ovn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230672    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230694    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60a8bc96-a399-4cd8-9ebf-a0499e48b232-serviceca\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230706    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230745    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pm9jz\" (UniqueName: \"kubernetes.io/projected/0ed8e1d1-8fb1-4fd7-b654-570714348084-kube-api-access-pm9jz\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230770    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysconfig\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-kubernetes\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.231844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230834    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230860    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-system-cni-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230884    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-modprobe-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230905    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-host\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230914    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230920    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/91344415-9650-42ef-a460-f8a3029a9100-system-cni-dir\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230928    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-netd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230954    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230978    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-cni-netd\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.230978    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-multus-certs\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231014    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-socket-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231046    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-registration-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231068    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-systemd\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231102    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-lib-modules\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231112    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5e22f156-742a-4317-aafc-f354d6f9a56c-ovnkube-script-lib\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231125    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:35.232628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231149    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-cnibin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231152    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/91344415-9650-42ef-a460-f8a3029a9100-cni-binary-copy\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231204    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb6vp\" (UniqueName: \"kubernetes.io/projected/7e942d48-6e53-498f-8d2b-b9f88ca36d83-kube-api-access-kb6vp\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5e22f156-742a-4317-aafc-f354d6f9a56c-host-run-ovn-kubernetes\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-tuned\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231254    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lm2z\" (UniqueName: \"kubernetes.io/projected/1c3ad984-92cf-4d56-8498-b42e467ff8e4-kube-api-access-7lm2z\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231278    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13910b21-8b7c-46fa-b444-194c280bd51c-agent-certs\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.233202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.231301    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-multus\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.233636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.233622    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5e22f156-742a-4317-aafc-f354d6f9a56c-ovn-node-metrics-cert\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.237297 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.237266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bpb58\" (UniqueName: \"kubernetes.io/projected/91344415-9650-42ef-a460-f8a3029a9100-kube-api-access-bpb58\") pod \"multus-additional-cni-plugins-chtg8\" (UID: \"91344415-9650-42ef-a460-f8a3029a9100\") " pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.237773 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.237756    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vqcn\" (UniqueName: \"kubernetes.io/projected/5e22f156-742a-4317-aafc-f354d6f9a56c-kube-api-access-6vqcn\") pod \"ovnkube-node-9fth2\" (UID: \"5e22f156-742a-4317-aafc-f354d6f9a56c\") " pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.264843 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.264823    2583 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:35.332378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332349    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13910b21-8b7c-46fa-b444-194c280bd51c-konnectivity-ca\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.332378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zspv\" (UniqueName: \"kubernetes.io/projected/60a8bc96-a399-4cd8-9ebf-a0499e48b232-kube-api-access-6zspv\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332395    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-daemon-config\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e942d48-6e53-498f-8d2b-b9f88ca36d83-hosts-file\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60a8bc96-a399-4cd8-9ebf-a0499e48b232-host\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332439    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-system-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332453    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-etc-kubernetes\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-sys\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332493    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-k8s-cni-cncf-io\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332519    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-bin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7e942d48-6e53-498f-8d2b-b9f88ca36d83-hosts-file\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332543    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-var-lib-kubelet\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332564    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-etc-kubernetes\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332586    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-sys\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332598    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-kubelet-config\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332612    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-var-lib-kubelet\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332618    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-k8s-cni-cncf-io\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332622    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-system-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/60a8bc96-a399-4cd8-9ebf-a0499e48b232-host\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.332651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-kubelet-config\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332655    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-bin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332733    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-dbus\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332757    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acf595f4-708b-416b-9dc3-166a6166a23e-host-slash\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332773    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332796    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47mxr\" (UniqueName: \"kubernetes.io/projected/22924846-24ee-409a-a0fe-2acdc375fa2b-kube-api-access-47mxr\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-tmp\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332827    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/7fad80d1-6663-4c00-a282-2815aad25ba7-dbus\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332837    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/acf595f4-708b-416b-9dc3-166a6166a23e-host-slash\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/acf595f4-708b-416b-9dc3-166a6166a23e-iptables-alerter-script\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332880    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99d4v\" (UniqueName: \"kubernetes.io/projected/acf595f4-708b-416b-9dc3-166a6166a23e-kube-api-access-99d4v\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e942d48-6e53-498f-8d2b-b9f88ca36d83-tmp-dir\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-device-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.332979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-device-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333047    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-daemon-config\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333069    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-etc-selinux\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.333476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333095    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-sys-fs\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60a8bc96-a399-4cd8-9ebf-a0499e48b232-serviceca\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333119    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-etc-selinux\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333128    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/13910b21-8b7c-46fa-b444-194c280bd51c-konnectivity-ca\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333149    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pm9jz\" (UniqueName: \"kubernetes.io/projected/0ed8e1d1-8fb1-4fd7-b654-570714348084-kube-api-access-pm9jz\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333164    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysconfig\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333164    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-sys-fs\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333179    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-kubernetes\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333212    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-modprobe-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-host\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysconfig\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333258    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-multus-certs\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333300    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-socket-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333320    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-registration-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333342    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-systemd\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-modprobe-d\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.333357    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:35.334410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333364    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-lib-modules\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333376    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-multus-certs\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333387    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333399    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-host\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-cnibin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333423    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/acf595f4-708b-416b-9dc3-166a6166a23e-iptables-alerter-script\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.333454    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:35.833413361 +0000 UTC m=+2.051329684 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333465    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-socket-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333279    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-kubernetes\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333464    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-registration-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333511    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb6vp\" (UniqueName: \"kubernetes.io/projected/7e942d48-6e53-498f-8d2b-b9f88ca36d83-kube-api-access-kb6vp\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333522    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-lib-modules\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-tuned\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333504    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-cnibin\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333545    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lm2z\" (UniqueName: \"kubernetes.io/projected/1c3ad984-92cf-4d56-8498-b42e467ff8e4-kube-api-access-7lm2z\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333600    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13910b21-8b7c-46fa-b444-194c280bd51c-agent-certs\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-multus\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.334948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333595    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-systemd\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333676    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-conf-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333646    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-conf-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333718    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-run\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-cni-binary-copy\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333788    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-run\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7e942d48-6e53-498f-8d2b-b9f88ca36d83-tmp-dir\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-netns\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-conf\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333871    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333815    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-cni-dir\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333742    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-cni-multus\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333894    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-kubelet\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333924    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-var-lib-kubelet\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333931    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-socket-dir-parent\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333961    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-hostroot\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333985    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.335437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9b46\" (UniqueName: \"kubernetes.io/projected/b10de668-df4d-40b5-9e19-255843710763-kube-api-access-r9b46\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334078    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-hostroot\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334088    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0ed8e1d1-8fb1-4fd7-b654-570714348084-kubelet-dir\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.334148    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.334199    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:35.83417652 +0000 UTC m=+2.052092831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.333988    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-multus-socket-dir-parent\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334236    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/22924846-24ee-409a-a0fe-2acdc375fa2b-host-run-netns\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334234    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22924846-24ee-409a-a0fe-2acdc375fa2b-cni-binary-copy\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334270    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-sysctl-conf\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.334402    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/60a8bc96-a399-4cd8-9ebf-a0499e48b232-serviceca\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.335718    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-etc-tuned\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.336186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.335767    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1c3ad984-92cf-4d56-8498-b42e467ff8e4-tmp\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.336513 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.336215    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/13910b21-8b7c-46fa-b444-194c280bd51c-agent-certs\") pod \"konnectivity-agent-ldrh2\" (UID: \"13910b21-8b7c-46fa-b444-194c280bd51c\") " pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.340688 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.340667    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:35.340688 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.340690    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:35.340817 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.340700    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:35.340817 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.340751    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:35.840740438 +0000 UTC m=+2.058656741 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:35.342890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.342864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zspv\" (UniqueName: \"kubernetes.io/projected/60a8bc96-a399-4cd8-9ebf-a0499e48b232-kube-api-access-6zspv\") pod \"node-ca-lt4dq\" (UID: \"60a8bc96-a399-4cd8-9ebf-a0499e48b232\") " pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.343610 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.343585    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99d4v\" (UniqueName: \"kubernetes.io/projected/acf595f4-708b-416b-9dc3-166a6166a23e-kube-api-access-99d4v\") pod \"iptables-alerter-s2hwk\" (UID: \"acf595f4-708b-416b-9dc3-166a6166a23e\") " pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.344236 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.344217    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pm9jz\" (UniqueName: \"kubernetes.io/projected/0ed8e1d1-8fb1-4fd7-b654-570714348084-kube-api-access-pm9jz\") pod \"aws-ebs-csi-driver-node-728r2\" (UID: \"0ed8e1d1-8fb1-4fd7-b654-570714348084\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.344311 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.344266    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47mxr\" (UniqueName: \"kubernetes.io/projected/22924846-24ee-409a-a0fe-2acdc375fa2b-kube-api-access-47mxr\") pod \"multus-94lfj\" (UID: \"22924846-24ee-409a-a0fe-2acdc375fa2b\") " pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.344399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.344385    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lm2z\" (UniqueName: \"kubernetes.io/projected/1c3ad984-92cf-4d56-8498-b42e467ff8e4-kube-api-access-7lm2z\") pod \"tuned-9ds6k\" (UID: \"1c3ad984-92cf-4d56-8498-b42e467ff8e4\") " pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.344598 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.344580    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9b46\" (UniqueName: \"kubernetes.io/projected/b10de668-df4d-40b5-9e19-255843710763-kube-api-access-r9b46\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.344899 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.344882    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb6vp\" (UniqueName: \"kubernetes.io/projected/7e942d48-6e53-498f-8d2b-b9f88ca36d83-kube-api-access-kb6vp\") pod \"node-resolver-ljkxh\" (UID: \"7e942d48-6e53-498f-8d2b-b9f88ca36d83\") " pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.393049 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.393013    2583 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:35.446820 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.446746    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:28:35.458921 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.458899    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-chtg8"
Sep 08 13:28:35.475613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.475588    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-94lfj"
Sep 08 13:28:35.488377 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.488354    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-s2hwk"
Sep 08 13:28:35.511196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.511039    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2"
Sep 08 13:28:35.521865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.521842    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:35.542469 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.542445    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-9ds6k"
Sep 08 13:28:35.557155 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.557132    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ljkxh"
Sep 08 13:28:35.561772 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.561750    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lt4dq"
Sep 08 13:28:35.564776 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.564745    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c3ad984_92cf_4d56_8498_b42e467ff8e4.slice/crio-e8e8a6fa0f9cb52d31f196a456910b3a80281ef2d41c52d86f324cd4334b5713 WatchSource:0}: Error finding container e8e8a6fa0f9cb52d31f196a456910b3a80281ef2d41c52d86f324cd4334b5713: Status 404 returned error can't find the container with id e8e8a6fa0f9cb52d31f196a456910b3a80281ef2d41c52d86f324cd4334b5713
Sep 08 13:28:35.565264 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.565246    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13910b21_8b7c_46fa_b444_194c280bd51c.slice/crio-c2665595814948cda73071271349209bed4ff3601dee70be5443c304e739bb9d WatchSource:0}: Error finding container c2665595814948cda73071271349209bed4ff3601dee70be5443c304e739bb9d: Status 404 returned error can't find the container with id c2665595814948cda73071271349209bed4ff3601dee70be5443c304e739bb9d
Sep 08 13:28:35.566469 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.566445    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cd9ea59b49ae21cabe35a064ca81a74.slice/crio-a68877a0f5ea234ce0b37c8a7d8268847c132d8ef84dd29b0b46fbd12ac24757 WatchSource:0}: Error finding container a68877a0f5ea234ce0b37c8a7d8268847c132d8ef84dd29b0b46fbd12ac24757: Status 404 returned error can't find the container with id a68877a0f5ea234ce0b37c8a7d8268847c132d8ef84dd29b0b46fbd12ac24757
Sep 08 13:28:35.569378 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.569268    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e942d48_6e53_498f_8d2b_b9f88ca36d83.slice/crio-2857f527b9abdd0e2e286d48820943c584f308cb7eea85511393217f12a0fb9f WatchSource:0}: Error finding container 2857f527b9abdd0e2e286d48820943c584f308cb7eea85511393217f12a0fb9f: Status 404 returned error can't find the container with id 2857f527b9abdd0e2e286d48820943c584f308cb7eea85511393217f12a0fb9f
Sep 08 13:28:35.570492 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.570472    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:28:35.572738 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.572686    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60a8bc96_a399_4cd8_9ebf_a0499e48b232.slice/crio-e2be1ced77a11383ff24d6e96c0d6e1fae3992173a94ead1177c6e89033df3d4 WatchSource:0}: Error finding container e2be1ced77a11383ff24d6e96c0d6e1fae3992173a94ead1177c6e89033df3d4: Status 404 returned error can't find the container with id e2be1ced77a11383ff24d6e96c0d6e1fae3992173a94ead1177c6e89033df3d4
Sep 08 13:28:35.837338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.837309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:35.837517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.837362    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:35.837517 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.837462    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:35.837625 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.837522    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:36.837507314 +0000 UTC m=+3.055423618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:35.837625 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.837463    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:35.837625 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.837610    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:36.837597298 +0000 UTC m=+3.055513603 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:35.937759 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:35.937717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:35.937928 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.937867    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:35.937928 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.937883    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:35.937928 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.937893    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:35.938032 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:35.937947    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:36.937931342 +0000 UTC m=+3.155847647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:35.952796 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.952770    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacf595f4_708b_416b_9dc3_166a6166a23e.slice/crio-66e54e2ea1637c5dff56af592cb0fe70fc50a0f68765fe1fa63843f6aac310c3 WatchSource:0}: Error finding container 66e54e2ea1637c5dff56af592cb0fe70fc50a0f68765fe1fa63843f6aac310c3: Status 404 returned error can't find the container with id 66e54e2ea1637c5dff56af592cb0fe70fc50a0f68765fe1fa63843f6aac310c3
Sep 08 13:28:35.987380 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:35.987347    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e22f156_742a_4317_aafc_f354d6f9a56c.slice/crio-ed7ee2038cfba9cb7ced8004a0dfbbcdca86dfbf6aba9a3f297d96269901cb5c WatchSource:0}: Error finding container ed7ee2038cfba9cb7ced8004a0dfbbcdca86dfbf6aba9a3f297d96269901cb5c: Status 404 returned error can't find the container with id ed7ee2038cfba9cb7ced8004a0dfbbcdca86dfbf6aba9a3f297d96269901cb5c
Sep 08 13:28:36.010482 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.010412    2583 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:36.165032 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.164961    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:23:35 +0000 UTC" deadline="2028-06-19 05:02:48.290855896 +0000 UTC"
Sep 08 13:28:36.165032 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.165028    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15591h34m12.125832422s"
Sep 08 13:28:36.230676 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.230606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ldrh2" event={"ID":"13910b21-8b7c-46fa-b444-194c280bd51c","Type":"ContainerStarted","Data":"c2665595814948cda73071271349209bed4ff3601dee70be5443c304e739bb9d"}
Sep 08 13:28:36.232079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.232036    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9ds6k" event={"ID":"1c3ad984-92cf-4d56-8498-b42e467ff8e4","Type":"ContainerStarted","Data":"e8e8a6fa0f9cb52d31f196a456910b3a80281ef2d41c52d86f324cd4334b5713"}
Sep 08 13:28:36.234353 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.234322    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"ed7ee2038cfba9cb7ced8004a0dfbbcdca86dfbf6aba9a3f297d96269901cb5c"}
Sep 08 13:28:36.237714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.237684    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s2hwk" event={"ID":"acf595f4-708b-416b-9dc3-166a6166a23e","Type":"ContainerStarted","Data":"66e54e2ea1637c5dff56af592cb0fe70fc50a0f68765fe1fa63843f6aac310c3"}
Sep 08 13:28:36.239478 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.239430    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lt4dq" event={"ID":"60a8bc96-a399-4cd8-9ebf-a0499e48b232","Type":"ContainerStarted","Data":"e2be1ced77a11383ff24d6e96c0d6e1fae3992173a94ead1177c6e89033df3d4"}
Sep 08 13:28:36.244805 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.244772    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljkxh" event={"ID":"7e942d48-6e53-498f-8d2b-b9f88ca36d83","Type":"ContainerStarted","Data":"2857f527b9abdd0e2e286d48820943c584f308cb7eea85511393217f12a0fb9f"}
Sep 08 13:28:36.246691 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.246639    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"8cd9ea59b49ae21cabe35a064ca81a74","Type":"ContainerStarted","Data":"a68877a0f5ea234ce0b37c8a7d8268847c132d8ef84dd29b0b46fbd12ac24757"}
Sep 08 13:28:36.845771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.845734    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:36.846016 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.845801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:36.846016 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.845913    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:36.846016 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.845971    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:38.845951828 +0000 UTC m=+5.063868136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:36.846195 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.846033    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:36.846195 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.846069    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:38.846056612 +0000 UTC m=+5.063972919 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:36.946941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:36.946898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:36.947139 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.947093    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:36.947139 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.947112    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:36.947139 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.947124    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:36.947314 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:36.947194    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:38.94717571 +0000 UTC m=+5.165092030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:37.165827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.165735    2583 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-07 13:23:35 +0000 UTC" deadline="2028-03-27 23:51:19.231534416 +0000 UTC"
Sep 08 13:28:37.165827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.165776    2583 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13594h22m42.065764634s"
Sep 08 13:28:37.224514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.224482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:37.224791 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:37.224745    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:37.224876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.224784    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:37.224935 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:37.224912    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:37.225003 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.224945    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:37.225123 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:37.225074    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:37.633094 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:37.632990    2583 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 08 13:28:38.864615 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:38.864580    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:38.865187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:38.864637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:38.865187 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.864766    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:38.865187 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.864848    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.864826263 +0000 UTC m=+9.082742588 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:38.865187 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.864772    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:38.865187 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.864927    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.864909966 +0000 UTC m=+9.082826287 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:38.965697 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:38.965627    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:38.965888 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.965825    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:38.965888 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.965854    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:38.965888 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.965867    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:38.966045 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:38.965929    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:42.965909908 +0000 UTC m=+9.183826230 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:39.225348 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:39.225257    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:39.225348 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:39.225274    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:39.225598 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:39.225405    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:39.225758 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:39.225729    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:39.225891 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:39.225846    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:39.225963 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:39.225912    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:39.340697 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:39.340471    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91344415_9650_42ef_a460_f8a3029a9100.slice/crio-d15d0d3e9cd7999cf8d1f8b73eebd1408af76b62bbc013be9278de7f3a5590b7 WatchSource:0}: Error finding container d15d0d3e9cd7999cf8d1f8b73eebd1408af76b62bbc013be9278de7f3a5590b7: Status 404 returned error can't find the container with id d15d0d3e9cd7999cf8d1f8b73eebd1408af76b62bbc013be9278de7f3a5590b7
Sep 08 13:28:39.402483 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:39.402445    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed8e1d1_8fb1_4fd7_b654_570714348084.slice/crio-b75b8889b29322545157190bbfa07419c0cac49afddd31d928e49af2b1a40b66 WatchSource:0}: Error finding container b75b8889b29322545157190bbfa07419c0cac49afddd31d928e49af2b1a40b66: Status 404 returned error can't find the container with id b75b8889b29322545157190bbfa07419c0cac49afddd31d928e49af2b1a40b66
Sep 08 13:28:40.269234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.269150    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lt4dq" event={"ID":"60a8bc96-a399-4cd8-9ebf-a0499e48b232","Type":"ContainerStarted","Data":"2de3b0bd5ba1fe1e30bb71aad10f0f18b8028250406869a6e1a03390e63d3998"}
Sep 08 13:28:40.272750 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.272256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ljkxh" event={"ID":"7e942d48-6e53-498f-8d2b-b9f88ca36d83","Type":"ContainerStarted","Data":"4e8e9ad5bf14fa73e66827bf18a4207f81a7af88b7ea85f96cef4417f419f79c"}
Sep 08 13:28:40.274718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.274666    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94lfj" event={"ID":"22924846-24ee-409a-a0fe-2acdc375fa2b","Type":"ContainerStarted","Data":"e1d5904e3688843b81c84ca5030e6fe6ab2276500bf446da729305260a366338"}
Sep 08 13:28:40.279427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.279348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerStarted","Data":"d15d0d3e9cd7999cf8d1f8b73eebd1408af76b62bbc013be9278de7f3a5590b7"}
Sep 08 13:28:40.283909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.283876    2583 generic.go:356] "Generic (PLEG): container finished" podID="8cd9ea59b49ae21cabe35a064ca81a74" containerID="d8b374b2ee21467ec8f3b73a628fc93eae65bbff870fd7281034b6cedc822607" exitCode=0
Sep 08 13:28:40.284050 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.283979    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"8cd9ea59b49ae21cabe35a064ca81a74","Type":"ContainerDied","Data":"d8b374b2ee21467ec8f3b73a628fc93eae65bbff870fd7281034b6cedc822607"}
Sep 08 13:28:40.294991 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.294963    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ldrh2" event={"ID":"13910b21-8b7c-46fa-b444-194c280bd51c","Type":"ContainerStarted","Data":"1726a3cc6bbea2441aaef62fb27a3d54f520a811dbf179972a0f0ab736b0b9fc"}
Sep 08 13:28:40.298574 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.298546    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2" event={"ID":"0ed8e1d1-8fb1-4fd7-b654-570714348084","Type":"ContainerStarted","Data":"b75b8889b29322545157190bbfa07419c0cac49afddd31d928e49af2b1a40b66"}
Sep 08 13:28:40.302693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.302463    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lt4dq" podStartSLOduration=2.5488466560000003 podStartE2EDuration="6.302422878s" totalImagesPullingTime="3.753576222s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:40.287612906 +0000 UTC m=+6.505529255" watchObservedRunningTime="2026-09-08 13:28:40.302422878 +0000 UTC m=+6.520339208"
Sep 08 13:28:40.318243 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.318190    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ljkxh" podStartSLOduration=2.558571446 podStartE2EDuration="6.318175581s" totalImagesPullingTime="3.759604135s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:40.302602121 +0000 UTC m=+6.520518464" watchObservedRunningTime="2026-09-08 13:28:40.318175581 +0000 UTC m=+6.536091910"
Sep 08 13:28:40.938962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.938892    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:40.939460 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.939440    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:40.955066 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:40.955006    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ldrh2" podStartSLOduration=3.204746583 podStartE2EDuration="6.95498818s" totalImagesPullingTime="3.750241597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:40.336041904 +0000 UTC m=+6.553958234" watchObservedRunningTime="2026-09-08 13:28:40.95498818 +0000 UTC m=+7.172904509"
Sep 08 13:28:41.224937 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.224854    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:41.225114 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:41.225001    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:41.225428 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.225407    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:41.225732 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:41.225512    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:41.225732 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.225587    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:41.225732 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:41.225685    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:41.307500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.307460    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" event={"ID":"8cd9ea59b49ae21cabe35a064ca81a74","Type":"ContainerStarted","Data":"bd64cebe8e7938b39be3ba5135fdc77b9716e6f391e92651bb414294d7e1d6d6"}
Sep 08 13:28:41.312124 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.312092    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-s2hwk" event={"ID":"acf595f4-708b-416b-9dc3-166a6166a23e","Type":"ContainerStarted","Data":"482154beeadffcee4b1fcccfe5e0516f3597792e42d90caf15fd3739969315c3"}
Sep 08 13:28:41.323740 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.323388    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-23.ec2.internal" podStartSLOduration=6.3233655859999995 podStartE2EDuration="6.323365586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:41.322814391 +0000 UTC m=+7.540730751" watchObservedRunningTime="2026-09-08 13:28:41.323365586 +0000 UTC m=+7.541281916"
Sep 08 13:28:41.339875 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:41.339156    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-s2hwk" podStartSLOduration=3.961689588 podStartE2EDuration="7.339135132s" totalImagesPullingTime="3.377445544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:41.338793643 +0000 UTC m=+7.556709990" watchObservedRunningTime="2026-09-08 13:28:41.339135132 +0000 UTC m=+7.557051462"
Sep 08 13:28:42.313460 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:42.313430    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 13:28:42.897935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:42.897883    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:42.897935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:42.897939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:42.898204 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.898066    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:42.898204 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.898110    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:50.898097287 +0000 UTC m=+17.116013592 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:42.898429 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.898408    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:42.898486 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.898450    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:50.898440173 +0000 UTC m=+17.116356477 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:42.998285 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:42.998250    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:42.998481 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.998455    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:42.998481 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.998473    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:42.998481 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.998482    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:42.998693 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:42.998523    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:28:50.998510306 +0000 UTC m=+17.216426608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:43.225586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:43.225482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:43.225756 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:43.225639    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:43.225756 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:43.225704    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:43.225923 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:43.225786    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:43.225923 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:43.225828    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:43.225923 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:43.225874    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:45.225163    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:45.225211    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:45.225344    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:45.225383    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:45.225529    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:45.225719 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:45.225630    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:47.225102 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:47.225066    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:47.225599 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:47.225079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:47.225599 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:47.225191    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:47.225599 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:47.225207    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:47.225599 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:47.225295    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:47.225599 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:47.225395    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:49.224525 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:49.224490    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:49.224941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:49.224531    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:49.224941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:49.224613    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:49.224941 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:49.224630    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:49.224941 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:49.224690    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:49.224941 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:49.224802    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:49.461861 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:49.461825    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:49.547812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:49.547727    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:50.959681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:50.959644    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:50.960143 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:50.959715    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:50.960143 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:50.959779    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:50.960143 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:50.959842    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:06.959819532 +0000 UTC m=+33.177735844 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:28:50.960143 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:50.959842    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:50.960143 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:50.959900    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:06.959881621 +0000 UTC m=+33.177797984 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:28:51.060746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:51.060716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:51.060915 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.060858    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:28:51.060915 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.060877    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:28:51.060915 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.060890    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:51.061058 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.060954    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:07.060934578 +0000 UTC m=+33.278850895 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:51.224720    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.224875    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:51.225186    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:51.225214    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.225292    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:51.225429 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:51.225358    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:53.225079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.225030    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:53.225079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.225064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:53.225630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.225064    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:53.225630 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:53.225151    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:53.225630 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:53.225259    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:53.225630 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:53.225347    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:53.372239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.372207    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:53.372410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.372342    2583 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 08 13:28:53.372870 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.372844    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ldrh2"
Sep 08 13:28:53.439152 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.439111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:53.852765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.852720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:53.902269 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:53.902238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:54.163568 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:28:54.163527    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ce096f9d3777e9f36fc81bb4086cc7.slice/crio-56a9f8bfc0a1c4972a80c6d270b61993a657d30189a4d7e800ead362bb67ac8f WatchSource:0}: Error finding container 56a9f8bfc0a1c4972a80c6d270b61993a657d30189a4d7e800ead362bb67ac8f: Status 404 returned error can't find the container with id 56a9f8bfc0a1c4972a80c6d270b61993a657d30189a4d7e800ead362bb67ac8f
Sep 08 13:28:54.346799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:54.346632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2" event={"ID":"0ed8e1d1-8fb1-4fd7-b654-570714348084","Type":"ContainerStarted","Data":"eaaafd8ce35659290c0c14f4ea031a4baec56ec2f6980097784dabe3d4344a99"}
Sep 08 13:28:54.348175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:54.348141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerStarted","Data":"76fe6d8963aab279de4861ad664b1a11689dabfdd1da8e3e7681a2aafa7efa32"}
Sep 08 13:28:54.349225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:54.349205    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" event={"ID":"b8ce096f9d3777e9f36fc81bb4086cc7","Type":"ContainerStarted","Data":"56a9f8bfc0a1c4972a80c6d270b61993a657d30189a4d7e800ead362bb67ac8f"}
Sep 08 13:28:55.225115 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.224859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:55.225272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.224903    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:55.225272 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:55.225200    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:55.225373 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:55.225303    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:55.225373 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.224921    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:55.225454 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:55.225431    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:55.354328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354304    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354719    2583 generic.go:356] "Generic (PLEG): container finished" podID="5e22f156-742a-4317-aafc-f354d6f9a56c" containerID="192d37967ac677ab82c3d5d0f4832c6c4fa6bda38b8ba236f45f1ae368c9c2ca" exitCode=1
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354793    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"ed1d5f5bf8852a29a86287ae6c0860543922742ea379aa59c3e6d049001b7328"}
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354884    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"0af718fda1e659d7ca0a10e15e398957bd52a0a7a4e27485b8d371fbd65cf91b"}
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"e92c3e0bcc2f12b29ddc649960da22074af9bee1e195c6e9f3f4bf58773baec8"}
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.354975    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"bc6a72549bcb5f8728e2ec9d5505d5e8ed269e69eb06430bf19d1e67a535077f"}
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.355008    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerDied","Data":"192d37967ac677ab82c3d5d0f4832c6c4fa6bda38b8ba236f45f1ae368c9c2ca"}
Sep 08 13:28:55.355120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.355045    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"0eeced1bc85d50a74fa0579655f5008b2828eea5d8b568949dbc2ca44b6ec98f"}
Sep 08 13:28:55.356411 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.356381    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-9ds6k" event={"ID":"1c3ad984-92cf-4d56-8498-b42e467ff8e4","Type":"ContainerStarted","Data":"375905a73ea3193091d9ba7b1b79a794d22c10cf2af8505c398583a0aef752fa"}
Sep 08 13:28:55.357990 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.357968    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-94lfj" event={"ID":"22924846-24ee-409a-a0fe-2acdc375fa2b","Type":"ContainerStarted","Data":"b17af888a26a5cb22653c25d7a0f4f28bf10de56b9b0f502f94082a28c5c74ca"}
Sep 08 13:28:55.359506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.359483    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="76fe6d8963aab279de4861ad664b1a11689dabfdd1da8e3e7681a2aafa7efa32" exitCode=0
Sep 08 13:28:55.359620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.359528    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"76fe6d8963aab279de4861ad664b1a11689dabfdd1da8e3e7681a2aafa7efa32"}
Sep 08 13:28:55.372606 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.372546    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-9ds6k" podStartSLOduration=2.7843176979999997 podStartE2EDuration="21.372534415s" totalImagesPullingTime="18.588216717s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:55.372213072 +0000 UTC m=+21.590129422" watchObservedRunningTime="2026-09-08 13:28:55.372534415 +0000 UTC m=+21.590450740"
Sep 08 13:28:55.405651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.405605    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-94lfj" podStartSLOduration=6.425147141 podStartE2EDuration="21.405590885s" totalImagesPullingTime="14.980443744s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:55.405158907 +0000 UTC m=+21.623075231" watchObservedRunningTime="2026-09-08 13:28:55.405590885 +0000 UTC m=+21.623507209"
Sep 08 13:28:55.568387 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.568351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:55.670294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.670260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:55.696707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:55.696685    2583 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 08 13:28:56.192467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:56.192369    2583 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-08T13:28:55.696700522Z","UUID":"b4b9c0ab-b2b4-4932-82d3-2f26bf501ec7","Handler":null,"Name":"","Endpoint":""}
Sep 08 13:28:56.194117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:56.194095    2583 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 08 13:28:56.194225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:56.194131    2583 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 08 13:28:56.369568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:56.369506    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2" event={"ID":"0ed8e1d1-8fb1-4fd7-b654-570714348084","Type":"ContainerStarted","Data":"43a865d1081b6d8db31f5887a787cf89a25bb1519b88c49406d668761de18233"}
Sep 08 13:28:57.225445 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.225373    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:57.225607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.225376    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:57.225607 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:57.225474    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:57.225607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.225376    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:57.225607 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:57.225577    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:28:57.225730 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:57.225686    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:57.375777 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.375750    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:28:57.376316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.376224    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"b639934d42e2abcce04b687f92ebd0c8bec33325b32a4cd2580243e94b44cddb"}
Sep 08 13:28:57.378700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.378674    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2" event={"ID":"0ed8e1d1-8fb1-4fd7-b654-570714348084","Type":"ContainerStarted","Data":"67b807adb7cdb11484629f1aea8a3b68c0c4b6231fbdba3f83bdcf1f0b9eb987"}
Sep 08 13:28:57.396013 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.395964    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-728r2" podStartSLOduration=6.22491356 podStartE2EDuration="23.395948482s" totalImagesPullingTime="17.171034922s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:28:57.395540637 +0000 UTC m=+23.613456982" watchObservedRunningTime="2026-09-08 13:28:57.395948482 +0000 UTC m=+23.613864807"
Sep 08 13:28:57.460455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.460420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:57.544803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.544763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:57.647957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.647929    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:57.705082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:57.705044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:28:59.225459 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:59.225420    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:28:59.225890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:59.225542    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:28:59.225890 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:59.225573    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:28:59.225890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:28:59.225614    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:28:59.225890 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:59.225684    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:28:59.225890 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:28:59.225771    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:29:01.224677 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.224482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:01.225025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.224482    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:01.225025 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:01.224746    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:29:01.225025 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:01.224838    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:29:01.225025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.224530    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:01.225025 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:01.224917    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:29:01.390338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.390311    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:29:01.390725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.390700    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"a9df78423944bd1b5eb98331570354e52e10f27a9244cf0890b5ca2420d8120d"}
Sep 08 13:29:01.390979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.390952    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:01.391110 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.391097    2583 scope.go:117] "RemoveContainer" containerID="192d37967ac677ab82c3d5d0f4832c6c4fa6bda38b8ba236f45f1ae368c9c2ca"
Sep 08 13:29:01.392858 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.392834    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="96d161c574f6396182fbc2662fb4a14c9e9a0980ca474e10cd1296297e8cd971" exitCode=0
Sep 08 13:29:01.392958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.392911    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"96d161c574f6396182fbc2662fb4a14c9e9a0980ca474e10cd1296297e8cd971"}
Sep 08 13:29:01.394585 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.394538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" event={"ID":"b8ce096f9d3777e9f36fc81bb4086cc7","Type":"ContainerStarted","Data":"6eae9d92541b68b6f84c69fadbb3a79d6e16372f622dea3a0046247186e1f5dd"}
Sep 08 13:29:01.408295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.408273    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:01.427068 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.427027    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-23.ec2.internal" podStartSLOduration=26.427009166 podStartE2EDuration="26.427009166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:01.426728345 +0000 UTC m=+27.644644670" watchObservedRunningTime="2026-09-08 13:29:01.427009166 +0000 UTC m=+27.644925487"
Sep 08 13:29:01.570052 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.570000    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:01.621748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:01.621716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:02.402112 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.402083    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:29:02.402927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.402506    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" event={"ID":"5e22f156-742a-4317-aafc-f354d6f9a56c","Type":"ContainerStarted","Data":"8a1b2f0fd170c6b751719b01f328f3d324565963062fee39498b1dac92ea64a6"}
Sep 08 13:29:02.402927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.402869    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:02.403043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.402996    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:02.420405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.420371    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:02.430396 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.430353    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2" podStartSLOduration=10.253521558 podStartE2EDuration="28.430339824s" totalImagesPullingTime="18.176818266s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:02.429309301 +0000 UTC m=+28.647225637" watchObservedRunningTime="2026-09-08 13:29:02.430339824 +0000 UTC m=+28.648256149"
Sep 08 13:29:02.653607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.653500    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ngbdv"]
Sep 08 13:29:02.653753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.653703    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:02.653857 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:02.653832    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:29:02.654292 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.654259    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zjktc"]
Sep 08 13:29:02.654393 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.654354    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:02.654465 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:02.654439    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:29:02.655562 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.655527    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-msdpw"]
Sep 08 13:29:02.655641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:02.655634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:02.655726 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:02.655709    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:29:03.407570 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:03.407461    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="6b4622ae042089251ef693b4406c9da66018c71ed1bf2241b4d9678e6495d5ca" exitCode=0
Sep 08 13:29:03.407570 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:03.407541    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"6b4622ae042089251ef693b4406c9da66018c71ed1bf2241b4d9678e6495d5ca"}
Sep 08 13:29:03.613313 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:03.613107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:04.225862 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:04.225773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:04.226018 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:04.225873    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:04.226018 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:04.225912    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:04.226018 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:04.225909    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:29:04.226018 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:04.225999    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:29:04.226212 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:04.226103    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:29:05.321736 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:05.321701    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:06.228215 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:06.228185    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:06.228406 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:06.228184    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:06.228406 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.228297    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-zjktc" podUID="838f875f-9203-4a37-91fc-58c8564ab496"
Sep 08 13:29:06.228406 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:06.228185    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:06.228406 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.228353    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-msdpw" podUID="b10de668-df4d-40b5-9e19-255843710763"
Sep 08 13:29:06.228637 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.228407    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-ngbdv" podUID="7fad80d1-6663-4c00-a282-2815aad25ba7"
Sep 08 13:29:06.972440 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:06.972402    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:06.972902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:06.972462    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:06.972902 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.972567    2583 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 08 13:29:06.972902 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.972568    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:29:06.972902 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.972616    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret podName:7fad80d1-6663-4c00-a282-2815aad25ba7 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:38.97260448 +0000 UTC m=+65.190520783 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret") pod "global-pull-secret-syncer-ngbdv" (UID: "7fad80d1-6663-4c00-a282-2815aad25ba7") : object "kube-system"/"original-pull-secret" not registered
Sep 08 13:29:06.972902 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:06.972630    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:38.972623473 +0000 UTC m=+65.190539775 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 08 13:29:07.073257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.073173    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:07.073401 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.073370    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 08 13:29:07.073401 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.073393    2583 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 08 13:29:07.073486 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.073403    2583 projected.go:196] Error preparing data for projected volume kube-api-access-mvgsh for pod openshift-network-diagnostics/network-check-target-zjktc: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:29:07.073486 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.073457    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh podName:838f875f-9203-4a37-91fc-58c8564ab496 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:39.073442181 +0000 UTC m=+65.291358499 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-mvgsh" (UniqueName: "kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh") pod "network-check-target-zjktc" (UID: "838f875f-9203-4a37-91fc-58c8564ab496") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 08 13:29:07.097587 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.097542    2583 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-23.ec2.internal" event="NodeReady"
Sep 08 13:29:07.097748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.097703    2583 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 08 13:29:07.140864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.140831    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g2j96"]
Sep 08 13:29:07.146804 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.146777    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-5lhct"]
Sep 08 13:29:07.146942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.146913    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.148806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.148785    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.149227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.149209    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwgzv\""
Sep 08 13:29:07.149345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.149326    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 08 13:29:07.149405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.149363    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 08 13:29:07.150872 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.150849    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 08 13:29:07.151015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.150990    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 08 13:29:07.151101 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.151020    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rbspx\""
Sep 08 13:29:07.151101 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.151095    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 08 13:29:07.154683 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.154662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2j96"]
Sep 08 13:29:07.156733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.156495    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5lhct"]
Sep 08 13:29:07.275116 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.275263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275122    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfmc2\" (UniqueName: \"kubernetes.io/projected/0e7a9712-c9fa-4a81-8bee-304a282f05b6-kube-api-access-hfmc2\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.275263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7a9712-c9fa-4a81-8bee-304a282f05b6-config-volume\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.275263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1149f2f0-504e-4f23-9a7e-893aa93405d7-cert\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.275364 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqc9h\" (UniqueName: \"kubernetes.io/projected/1149f2f0-504e-4f23-9a7e-893aa93405d7-kube-api-access-lqc9h\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.275364 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.275298    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7a9712-c9fa-4a81-8bee-304a282f05b6-tmp-dir\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.375814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.375782    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqc9h\" (UniqueName: \"kubernetes.io/projected/1149f2f0-504e-4f23-9a7e-893aa93405d7-kube-api-access-lqc9h\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.375957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.375835    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7a9712-c9fa-4a81-8bee-304a282f05b6-tmp-dir\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.375998 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.375964    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.376054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.376004    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfmc2\" (UniqueName: \"kubernetes.io/projected/0e7a9712-c9fa-4a81-8bee-304a282f05b6-kube-api-access-hfmc2\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.376054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.376033    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7a9712-c9fa-4a81-8bee-304a282f05b6-config-volume\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.376054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.376049    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1149f2f0-504e-4f23-9a7e-893aa93405d7-cert\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.376186 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.376094    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:07.376186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.376103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0e7a9712-c9fa-4a81-8bee-304a282f05b6-tmp-dir\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.376186 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.376163    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:07.876142037 +0000 UTC m=+34.094058364 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:07.376607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.376588    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0e7a9712-c9fa-4a81-8bee-304a282f05b6-config-volume\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.380083 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.380067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1149f2f0-504e-4f23-9a7e-893aa93405d7-cert\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.384230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.384211    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfmc2\" (UniqueName: \"kubernetes.io/projected/0e7a9712-c9fa-4a81-8bee-304a282f05b6-kube-api-access-hfmc2\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.384801 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.384784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqc9h\" (UniqueName: \"kubernetes.io/projected/1149f2f0-504e-4f23-9a7e-893aa93405d7-kube-api-access-lqc9h\") pod \"ingress-canary-5lhct\" (UID: \"1149f2f0-504e-4f23-9a7e-893aa93405d7\") " pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.418408 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.418381    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="c8155f87ac5758f78e9c31e6a66fdb40370693f66a9ca83f2a391252f4c180cc" exitCode=0
Sep 08 13:29:07.418517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.418424    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"c8155f87ac5758f78e9c31e6a66fdb40370693f66a9ca83f2a391252f4c180cc"}
Sep 08 13:29:07.420802 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.420783    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:07.466850 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.466819    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-5lhct"
Sep 08 13:29:07.479976 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.479950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:07.739602 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.739507    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-5lhct"]
Sep 08 13:29:07.744577 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:29:07.744417    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1149f2f0_504e_4f23_9a7e_893aa93405d7.slice/crio-54f8fea478bb6cf5fe0af49da2da0eabf2b8e7518673880cb29749cfb5cab5c2 WatchSource:0}: Error finding container 54f8fea478bb6cf5fe0af49da2da0eabf2b8e7518673880cb29749cfb5cab5c2: Status 404 returned error can't find the container with id 54f8fea478bb6cf5fe0af49da2da0eabf2b8e7518673880cb29749cfb5cab5c2
Sep 08 13:29:07.880009 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:07.879966    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:07.880188 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.880100    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:07.880188 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:07.880150    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:08.880138309 +0000 UTC m=+35.098054613 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:08.224520 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.224484    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:08.224968 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.224484    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:08.224968 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.224499    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:08.227279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.227251    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:29:08.227431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.227255    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n94wj\""
Sep 08 13:29:08.227522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.227255    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56h6n\""
Sep 08 13:29:08.228218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.228005    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:29:08.228218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.228055    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:29:08.228218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.228009    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:29:08.422985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.422951    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5lhct" event={"ID":"1149f2f0-504e-4f23-9a7e-893aa93405d7","Type":"ContainerStarted","Data":"54f8fea478bb6cf5fe0af49da2da0eabf2b8e7518673880cb29749cfb5cab5c2"}
Sep 08 13:29:08.888072 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:08.888038    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:08.888317 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:08.888157    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:08.888317 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:08.888214    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:10.888197702 +0000 UTC m=+37.106114006 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:09.667218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:09.666991    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:09.725514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:09.725464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:10.904170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:10.904130    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:10.904710 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:10.904298    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:10.904710 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:10.904372    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:14.904354505 +0000 UTC m=+41.122270808 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:12.538636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:12.538603    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:12.640467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:12.640437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:13.439982 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:13.439943    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="16c6f46bb6cfa1112c0284b23202c2b082dc0509e32801c1e3b00c6fc08f380e" exitCode=0
Sep 08 13:29:13.440172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:13.440029    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"16c6f46bb6cfa1112c0284b23202c2b082dc0509e32801c1e3b00c6fc08f380e"}
Sep 08 13:29:13.441508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:13.441471    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-5lhct" event={"ID":"1149f2f0-504e-4f23-9a7e-893aa93405d7","Type":"ContainerStarted","Data":"48d7a392e0ff20689654eff2256d7ac0a6ce8f6b9104020b02b714f9e7716c9b"}
Sep 08 13:29:13.480793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:13.480739    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-5lhct" podStartSLOduration=1.8583900089999998 podStartE2EDuration="6.48071985s" totalImagesPullingTime="4.622329841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:13.48018093 +0000 UTC m=+39.698097255" watchObservedRunningTime="2026-09-08 13:29:13.48071985 +0000 UTC m=+39.698636179"
Sep 08 13:29:14.446985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:14.446948    2583 generic.go:356] "Generic (PLEG): container finished" podID="91344415-9650-42ef-a460-f8a3029a9100" containerID="40b530c757bc2136501305f964577b4502bfc3870a22ae209a741c14a87560db" exitCode=0
Sep 08 13:29:14.447505 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:14.447033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerDied","Data":"40b530c757bc2136501305f964577b4502bfc3870a22ae209a741c14a87560db"}
Sep 08 13:29:14.934205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:14.934163    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:14.934388 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:14.934278    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:14.934388 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:14.934331    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:22.934317995 +0000 UTC m=+49.152234299 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:15.453056 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:15.453011    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-chtg8" event={"ID":"91344415-9650-42ef-a460-f8a3029a9100","Type":"ContainerStarted","Data":"e928e70b6e300e351aee3524d27ad1edc89c8fd8d0131d5023a7466d2be1ab3f"}
Sep 08 13:29:15.473851 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:15.473807    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-chtg8" podStartSLOduration=11.461232622 podStartE2EDuration="41.473788331s" totalImagesPullingTime="30.012555709s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:15.472648813 +0000 UTC m=+41.690565150" watchObservedRunningTime="2026-09-08 13:29:15.473788331 +0000 UTC m=+41.691704659"
Sep 08 13:29:15.762000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:15.761908    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:15.816436 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:15.816401    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:22.992937 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:22.992900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:22.993350 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:22.993011    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:22.993350 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:22.993070    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:29:38.993056745 +0000 UTC m=+65.210973048 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:34.420923 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:34.420896    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9fth2"
Sep 08 13:29:34.572827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:34.572795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:34.673194 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:34.673108    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:35.523696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:35.523660    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:35.584627 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:35.584594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:39.008586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.008515    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:29:39.009082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.008603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:29:39.009082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.008654    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:39.009082 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:39.008895    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:29:39.009082 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:39.008961    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:11.008942134 +0000 UTC m=+97.226858437 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:29:39.011588 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.011542    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 08 13:29:39.011708 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.011674    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 08 13:29:39.018905 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:39.018881    2583 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 08 13:29:39.019030 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:29:39.018943    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs podName:b10de668-df4d-40b5-9e19-255843710763 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:43.018929142 +0000 UTC m=+129.236845445 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs") pod "network-metrics-daemon-msdpw" (UID: "b10de668-df4d-40b5-9e19-255843710763") : secret "metrics-daemon-secret" not found
Sep 08 13:29:39.022056 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.022033    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/7fad80d1-6663-4c00-a282-2815aad25ba7-original-pull-secret\") pod \"global-pull-secret-syncer-ngbdv\" (UID: \"7fad80d1-6663-4c00-a282-2815aad25ba7\") " pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:39.109605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.109534    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:39.112157 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.112135    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 08 13:29:39.121904 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.121888    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 08 13:29:39.132994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.132969    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvgsh\" (UniqueName: \"kubernetes.io/projected/838f875f-9203-4a37-91fc-58c8564ab496-kube-api-access-mvgsh\") pod \"network-check-target-zjktc\" (UID: \"838f875f-9203-4a37-91fc-58c8564ab496\") " pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:39.145379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.145355    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-n94wj\""
Sep 08 13:29:39.153749 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.153731    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:39.155761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.155745    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-ngbdv"
Sep 08 13:29:39.433918 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.433877    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-ngbdv"]
Sep 08 13:29:39.434909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.434818    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-zjktc"]
Sep 08 13:29:39.437448 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:29:39.437419    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fad80d1_6663_4c00_a282_2815aad25ba7.slice/crio-1a0604c01333527b66588ea5da2184ffac066c53bcee762c65e960ea8cb3dc20 WatchSource:0}: Error finding container 1a0604c01333527b66588ea5da2184ffac066c53bcee762c65e960ea8cb3dc20: Status 404 returned error can't find the container with id 1a0604c01333527b66588ea5da2184ffac066c53bcee762c65e960ea8cb3dc20
Sep 08 13:29:39.438157 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:29:39.438123    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod838f875f_9203_4a37_91fc_58c8564ab496.slice/crio-d854f988d45dd626b9c5d10139c2ddebe4059c79ccfc466d0f1eb16e3686dc36 WatchSource:0}: Error finding container d854f988d45dd626b9c5d10139c2ddebe4059c79ccfc466d0f1eb16e3686dc36: Status 404 returned error can't find the container with id d854f988d45dd626b9c5d10139c2ddebe4059c79ccfc466d0f1eb16e3686dc36
Sep 08 13:29:39.506043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.505996    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ngbdv" event={"ID":"7fad80d1-6663-4c00-a282-2815aad25ba7","Type":"ContainerStarted","Data":"1a0604c01333527b66588ea5da2184ffac066c53bcee762c65e960ea8cb3dc20"}
Sep 08 13:29:39.507125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:39.507100    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zjktc" event={"ID":"838f875f-9203-4a37-91fc-58c8564ab496","Type":"ContainerStarted","Data":"d854f988d45dd626b9c5d10139c2ddebe4059c79ccfc466d0f1eb16e3686dc36"}
Sep 08 13:29:45.814944 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:45.814896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:29:47.529969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:47.529933    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-ngbdv" event={"ID":"7fad80d1-6663-4c00-a282-2815aad25ba7","Type":"ContainerStarted","Data":"d6c4c0465587279fc3b9e42432675a9bf25868b3e4651c1f40ca119f9f6a34ae"}
Sep 08 13:29:47.531509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:47.531485    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-zjktc" event={"ID":"838f875f-9203-4a37-91fc-58c8564ab496","Type":"ContainerStarted","Data":"5eb2868a75ff03cebfc0a5c62190ef21a10b6af775ec8961247f49ab9db72da7"}
Sep 08 13:29:47.531644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:47.531608    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:29:47.544825 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:47.544779    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-ngbdv" podStartSLOduration=65.807181628 podStartE2EDuration="1m13.544766609s" totalImagesPullingTime="7.737584981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:47.543962302 +0000 UTC m=+73.761878638" watchObservedRunningTime="2026-09-08 13:29:47.544766609 +0000 UTC m=+73.762682933"
Sep 08 13:29:47.559526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:29:47.559480    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-zjktc" podStartSLOduration=65.824608797 podStartE2EDuration="1m13.559462353s" totalImagesPullingTime="7.734853556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:29:47.559015992 +0000 UTC m=+73.776932317" watchObservedRunningTime="2026-09-08 13:29:47.559462353 +0000 UTC m=+73.777378680"
Sep 08 13:30:11.016304 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:11.016177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:30:11.016304 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:11.016283    2583 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 08 13:30:11.016821 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:11.016333    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls podName:0e7a9712-c9fa-4a81-8bee-304a282f05b6 nodeName:}" failed. No retries permitted until 2026-09-08 13:31:15.016320262 +0000 UTC m=+161.234236566 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls") pod "dns-default-g2j96" (UID: "0e7a9712-c9fa-4a81-8bee-304a282f05b6") : secret "dns-default-metrics-tls" not found
Sep 08 13:30:15.172045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:15.172005    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:18.048093 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:18.048056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:18.536309 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:18.536283    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-zjktc"
Sep 08 13:30:21.308113 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.308077    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"]
Sep 08 13:30:21.342907 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.342875    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zwcmh"]
Sep 08 13:30:21.343084 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.343012    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.345899 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.345876    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Sep 08 13:30:21.346034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.345879    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Sep 08 13:30:21.346034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.345880    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.346034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.345924    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-ddcrl\""
Sep 08 13:30:21.346034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.345965    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.368831 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.368800    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-qxfbv"]
Sep 08 13:30:21.368942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.368853    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.370968 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.370947    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 08 13:30:21.371215 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.371201    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 08 13:30:21.371294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.371277    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 08 13:30:21.371562 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.371535    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 08 13:30:21.371608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.371544    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-7cs7j\""
Sep 08 13:30:21.376801 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.376782    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 08 13:30:21.384793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.384757    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-qxfbv"]
Sep 08 13:30:21.384793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.384785    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"]
Sep 08 13:30:21.384793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.384799    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zwcmh"]
Sep 08 13:30:21.385025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.384869    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv"
Sep 08 13:30:21.386689 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.386670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9be3d96-349b-465c-af4f-5aeac1612d32-serving-cert\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.386765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.386695    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cn8t\" (UniqueName: \"kubernetes.io/projected/a9be3d96-349b-465c-af4f-5aeac1612d32-kube-api-access-5cn8t\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.386765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.386724    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9be3d96-349b-465c-af4f-5aeac1612d32-config\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.387413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.387394    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-cb47r\""
Sep 08 13:30:21.400861 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.400838    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g5c67"]
Sep 08 13:30:21.421653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.421623    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:30:21.421793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.421748    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.424265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.424245    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jxxjh\""
Sep 08 13:30:21.424384 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.424261    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 08 13:30:21.424384 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.424271    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 08 13:30:21.443767 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.443739    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g5c67"]
Sep 08 13:30:21.443767 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.443772    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:30:21.443919 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.443845    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.446515 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.446498    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 08 13:30:21.446690 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.446518    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 08 13:30:21.446864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.446848    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9cf6z\""
Sep 08 13:30:21.446928 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.446890    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 08 13:30:21.452314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.452297    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 08 13:30:21.487978 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.487953    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.487985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488020    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488035    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fg7g\" (UniqueName: \"kubernetes.io/projected/bfb83340-d2b2-4688-89c8-5e55a96c0d01-kube-api-access-5fg7g\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488105    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-nginx-conf\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.488240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488177    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9be3d96-349b-465c-af4f-5aeac1612d32-serving-cert\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.488240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488202    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cn8t\" (UniqueName: \"kubernetes.io/projected/a9be3d96-349b-465c-af4f-5aeac1612d32-kube-api-access-5cn8t\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.488240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488217    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-tmp\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488232    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.488414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488246    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcc4v\" (UniqueName: \"kubernetes.io/projected/157397df-8364-4d42-ac07-8fb890790d54-kube-api-access-pcc4v\") pod \"network-check-source-685456655f-qxfbv\" (UID: \"157397df-8364-4d42-ac07-8fb890790d54\") " pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv"
Sep 08 13:30:21.488414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488263    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-snapshots\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488415    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488439    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488465    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfb83340-d2b2-4688-89c8-5e55a96c0d01-serving-cert\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.488528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488487    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488528    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488572    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82zls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.488681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.488594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9be3d96-349b-465c-af4f-5aeac1612d32-config\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.489025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.489010    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a9be3d96-349b-465c-af4f-5aeac1612d32-config\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.490426 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.490409    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a9be3d96-349b-465c-af4f-5aeac1612d32-serving-cert\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.496660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.496641    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cn8t\" (UniqueName: \"kubernetes.io/projected/a9be3d96-349b-465c-af4f-5aeac1612d32-kube-api-access-5cn8t\") pod \"service-ca-operator-859f94c4b-7zh5r\" (UID: \"a9be3d96-349b-465c-af4f-5aeac1612d32\") " pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.576385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.576305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:21.589180 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589152    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589210    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.589258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589236    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.589397 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:21.589343    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:30:21.589397 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:21.589359    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8d5d74d6-xh8f4: secret "image-registry-tls" not found
Sep 08 13:30:21.589397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fg7g\" (UniqueName: \"kubernetes.io/projected/bfb83340-d2b2-4688-89c8-5e55a96c0d01-kube-api-access-5fg7g\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589542 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589411    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-nginx-conf\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.589542 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:21.589426    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls podName:3c611e37-7835-408e-a108-695d1e8b978e nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.08940743 +0000 UTC m=+108.307323755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls") pod "image-registry-5b8d5d74d6-xh8f4" (UID: "3c611e37-7835-408e-a108-695d1e8b978e") : secret "image-registry-tls" not found
Sep 08 13:30:21.589542 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-tmp\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589542 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589527    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcc4v\" (UniqueName: \"kubernetes.io/projected/157397df-8364-4d42-ac07-8fb890790d54-kube-api-access-pcc4v\") pod \"network-check-source-685456655f-qxfbv\" (UID: \"157397df-8364-4d42-ac07-8fb890790d54\") " pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589614    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589640    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-snapshots\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589663    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589687    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589717    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfb83340-d2b2-4688-89c8-5e55a96c0d01-serving-cert\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.589776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589743    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.590117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589778    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.590117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589808    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82zls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.590117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589915    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-service-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.590117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.589921    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-tmp\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.590117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590093    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-nginx-conf\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:21.590370 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:21.590179    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:21.590370 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:21.590228    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:22.090215575 +0000 UTC m=+108.308131878 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:21.590370 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590295    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/bfb83340-d2b2-4688-89c8-5e55a96c0d01-snapshots\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.590370 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590353    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfb83340-d2b2-4688-89c8-5e55a96c0d01-trusted-ca-bundle\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.590749 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590482    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.590901 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590880    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.591009 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.590990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.592019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.591995    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.592315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.592300    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bfb83340-d2b2-4688-89c8-5e55a96c0d01-serving-cert\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.592934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.592913    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.598827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.598799    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fg7g\" (UniqueName: \"kubernetes.io/projected/bfb83340-d2b2-4688-89c8-5e55a96c0d01-kube-api-access-5fg7g\") pod \"insights-operator-76bcd6ccf5-zwcmh\" (UID: \"bfb83340-d2b2-4688-89c8-5e55a96c0d01\") " pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.598904 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.598810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.599111 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.599096    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82zls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:21.599844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.599830    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcc4v\" (UniqueName: \"kubernetes.io/projected/157397df-8364-4d42-ac07-8fb890790d54-kube-api-access-pcc4v\") pod \"network-check-source-685456655f-qxfbv\" (UID: \"157397df-8364-4d42-ac07-8fb890790d54\") " pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv"
Sep 08 13:30:21.651535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.651498    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"
Sep 08 13:30:21.677204 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.677174    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh"
Sep 08 13:30:21.677985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.677957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:21.693208 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.693080    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv"
Sep 08 13:30:21.957159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.957067    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r"]
Sep 08 13:30:21.959918 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:21.959894    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9be3d96_349b_465c_af4f_5aeac1612d32.slice/crio-ea72cabc77ea43743c6aababcf6e70b5ab95d2e8422ad0ca60b8d00d3c01ef4a WatchSource:0}: Error finding container ea72cabc77ea43743c6aababcf6e70b5ab95d2e8422ad0ca60b8d00d3c01ef4a: Status 404 returned error can't find the container with id ea72cabc77ea43743c6aababcf6e70b5ab95d2e8422ad0ca60b8d00d3c01ef4a
Sep 08 13:30:21.993596 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:21.993580    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-76bcd6ccf5-zwcmh"]
Sep 08 13:30:21.995755 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:21.995727    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb83340_d2b2_4688_89c8_5e55a96c0d01.slice/crio-50f429c9dc290fa849c21ac0224577765bd896bc4e7750468835d75446fa8ddb WatchSource:0}: Error finding container 50f429c9dc290fa849c21ac0224577765bd896bc4e7750468835d75446fa8ddb: Status 404 returned error can't find the container with id 50f429c9dc290fa849c21ac0224577765bd896bc4e7750468835d75446fa8ddb
Sep 08 13:30:22.019476 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.019449    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-qxfbv"]
Sep 08 13:30:22.022148 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:22.022122    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod157397df_8364_4d42_ac07_8fb890790d54.slice/crio-585c3efb14c2171fa1d6c487a57aaec4b9f8c781837970de229b5f0ce206b815 WatchSource:0}: Error finding container 585c3efb14c2171fa1d6c487a57aaec4b9f8c781837970de229b5f0ce206b815: Status 404 returned error can't find the container with id 585c3efb14c2171fa1d6c487a57aaec4b9f8c781837970de229b5f0ce206b815
Sep 08 13:30:22.093729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.093693    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.093779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:22.093814    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:22.093832    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8d5d74d6-xh8f4: secret "image-registry-tls" not found
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:22.093865    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:22.093886    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls podName:3c611e37-7835-408e-a108-695d1e8b978e nodeName:}" failed. No retries permitted until 2026-09-08 13:30:23.093867483 +0000 UTC m=+109.311783788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls") pod "image-registry-5b8d5d74d6-xh8f4" (UID: "3c611e37-7835-408e-a108-695d1e8b978e") : secret "image-registry-tls" not found
Sep 08 13:30:22.094620 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:22.093900    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:23.093890793 +0000 UTC m=+109.311807096 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:22.626336 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.626149    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv" event={"ID":"157397df-8364-4d42-ac07-8fb890790d54","Type":"ContainerStarted","Data":"bc0de1454c313afb7351e72b47fab21456132a5497a72b0b3f5411ebba835c44"}
Sep 08 13:30:22.626336 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.626212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv" event={"ID":"157397df-8364-4d42-ac07-8fb890790d54","Type":"ContainerStarted","Data":"585c3efb14c2171fa1d6c487a57aaec4b9f8c781837970de229b5f0ce206b815"}
Sep 08 13:30:22.628368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.628341    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r" event={"ID":"a9be3d96-349b-465c-af4f-5aeac1612d32","Type":"ContainerStarted","Data":"ea72cabc77ea43743c6aababcf6e70b5ab95d2e8422ad0ca60b8d00d3c01ef4a"}
Sep 08 13:30:22.630027 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.629982    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh" event={"ID":"bfb83340-d2b2-4688-89c8-5e55a96c0d01","Type":"ContainerStarted","Data":"50f429c9dc290fa849c21ac0224577765bd896bc4e7750468835d75446fa8ddb"}
Sep 08 13:30:22.644022 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:22.643972    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-qxfbv" podStartSLOduration=1.643954806 podStartE2EDuration="1.643954806s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:22.643077788 +0000 UTC m=+108.860994147" watchObservedRunningTime="2026-09-08 13:30:22.643954806 +0000 UTC m=+108.861871135"
Sep 08 13:30:23.103040 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:23.102999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:23.103215 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:23.103110    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:23.103215 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:23.103159    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:30:23.103215 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:23.103185    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8d5d74d6-xh8f4: secret "image-registry-tls" not found
Sep 08 13:30:23.103358 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:23.103220    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:23.103358 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:23.103248    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls podName:3c611e37-7835-408e-a108-695d1e8b978e nodeName:}" failed. No retries permitted until 2026-09-08 13:30:25.103229151 +0000 UTC m=+111.321145478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls") pod "image-registry-5b8d5d74d6-xh8f4" (UID: "3c611e37-7835-408e-a108-695d1e8b978e") : secret "image-registry-tls" not found
Sep 08 13:30:23.103358 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:23.103275    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:25.10325783 +0000 UTC m=+111.321174136 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:23.701129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:23.701091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:23.755608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:23.755570    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:24.638503 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:24.638469    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh" event={"ID":"bfb83340-d2b2-4688-89c8-5e55a96c0d01","Type":"ContainerStarted","Data":"b1b5ba2b0bded6a61f076f278f6beb857c7eb8f0427a0c58cf9457e170f64af5"}
Sep 08 13:30:24.640054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:24.640031    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r" event={"ID":"a9be3d96-349b-465c-af4f-5aeac1612d32","Type":"ContainerStarted","Data":"d7c78423ec03666e40bb1a42fbeb8ba52c8a1324baba3daf3a5b178e3973575f"}
Sep 08 13:30:24.656248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:24.656166    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh" podStartSLOduration=1.253941967 podStartE2EDuration="3.656153458s" totalImagesPullingTime="2.402211491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:24.655289062 +0000 UTC m=+110.873205398" watchObservedRunningTime="2026-09-08 13:30:24.656153458 +0000 UTC m=+110.874069783"
Sep 08 13:30:24.670134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:24.670068    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r" podStartSLOduration=1.229047258 podStartE2EDuration="3.670047474s" totalImagesPullingTime="2.441000216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:24.668956179 +0000 UTC m=+110.886872547" watchObservedRunningTime="2026-09-08 13:30:24.670047474 +0000 UTC m=+110.887963797"
Sep 08 13:30:25.117437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.117398    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.117471    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.117564    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.117583    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8d5d74d6-xh8f4: secret "image-registry-tls" not found
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.117605    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.117636    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls podName:3c611e37-7835-408e-a108-695d1e8b978e nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.117621442 +0000 UTC m=+115.335537745 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls") pod "image-registry-5b8d5d74d6-xh8f4" (UID: "3c611e37-7835-408e-a108-695d1e8b978e") : secret "image-registry-tls" not found
Sep 08 13:30:25.117953 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.117655    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.117641272 +0000 UTC m=+115.335557574 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:25.487820 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.487724    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-wsjt9"]
Sep 08 13:30:25.490293 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.490272    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.492455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.492426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:25.492611 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.492515    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 08 13:30:25.492611 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.492506    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j78zr\""
Sep 08 13:30:25.493184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.493166    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 08 13:30:25.521354 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.521323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.521499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.521359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wkcrc\" (UniqueName: \"kubernetes.io/projected/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-api-access-wkcrc\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.521499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.521401    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4635088-a268-4e79-8ae3-ebaf77e812e1-data-volume\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.521499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.521417    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.521499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.521477    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4635088-a268-4e79-8ae3-ebaf77e812e1-crio-socket\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.538261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.538231    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsjt9"]
Sep 08 13:30:25.547309 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.547277    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:25.622803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.622767    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wkcrc\" (UniqueName: \"kubernetes.io/projected/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-api-access-wkcrc\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.622962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.622824    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4635088-a268-4e79-8ae3-ebaf77e812e1-data-volume\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.622962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.622936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.623035 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.623008    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4635088-a268-4e79-8ae3-ebaf77e812e1-crio-socket\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.623072 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.623052    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:25.623130 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.623092    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.623130 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.623127    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:26.123107811 +0000 UTC m=+112.341024117 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:25.623239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.623136    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d4635088-a268-4e79-8ae3-ebaf77e812e1-data-volume\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.623239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.623144    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d4635088-a268-4e79-8ae3-ebaf77e812e1-crio-socket\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.623239 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.623204    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:25.623350 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:25.623258    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:26.123243413 +0000 UTC m=+112.341159719 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:25.633290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.633269    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wkcrc\" (UniqueName: \"kubernetes.io/projected/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-api-access-wkcrc\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:25.965368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:25.965319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:26.017814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:26.017758    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:26.126739 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:26.126700    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:26.127156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:26.126776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:26.127156 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:26.126867    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:26.127156 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:26.126887    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:26.127156 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:26.126953    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:27.126933711 +0000 UTC m=+113.344850027 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:26.127156 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:26.126972    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:27.126963869 +0000 UTC m=+113.344880175 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:26.380230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:26.380188    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:27.136664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:27.136623    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:27.137103 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:27.136705    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:27.137103 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:27.136782    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:27.137103 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:27.136810    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:27.137103 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:27.136860    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.136847181 +0000 UTC m=+115.354763485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:27.137103 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:27.136873    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:29.136867303 +0000 UTC m=+115.354783606 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:27.240240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:27.240213    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljkxh_7e942d48-6e53-498f-8d2b-b9f88ca36d83/dns-node-resolver/0.log"
Sep 08 13:30:28.277884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.277851    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-75d99c484-7r842"]
Sep 08 13:30:28.280001 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.279986    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.282082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.282058    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 08 13:30:28.282823 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.282796    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 08 13:30:28.282929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.282850    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 08 13:30:28.282929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.282867    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 08 13:30:28.283037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.282980    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-pkhh5\""
Sep 08 13:30:28.288664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.288645    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-7r842"]
Sep 08 13:30:28.346869 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.346828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqj4c\" (UniqueName: \"kubernetes.io/projected/0d2e6582-d29c-421b-bf87-200717f8dd40-kube-api-access-mqj4c\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.347055 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.346931    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-key\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.347055 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.346968    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-cabundle\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.448339 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.448299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-key\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.448512 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.448345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-cabundle\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.448615 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.448539    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqj4c\" (UniqueName: \"kubernetes.io/projected/0d2e6582-d29c-421b-bf87-200717f8dd40-kube-api-access-mqj4c\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.448926 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.448907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-cabundle\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.450886 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.450864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0d2e6582-d29c-421b-bf87-200717f8dd40-signing-key\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.456889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.456863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqj4c\" (UniqueName: \"kubernetes.io/projected/0d2e6582-d29c-421b-bf87-200717f8dd40-kube-api-access-mqj4c\") pod \"service-ca-75d99c484-7r842\" (UID: \"0d2e6582-d29c-421b-bf87-200717f8dd40\") " pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.588857 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.588773    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-75d99c484-7r842"
Sep 08 13:30:28.640626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.640606    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lt4dq_60a8bc96-a399-4cd8-9ebf-a0499e48b232/node-ca/0.log"
Sep 08 13:30:28.858496 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:28.858407    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-75d99c484-7r842"]
Sep 08 13:30:28.862349 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:28.862319    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d2e6582_d29c_421b_bf87_200717f8dd40.slice/crio-75127ff174f070fd113f3278cd62b305707813bb4b36917d47f1d2e77c954dcc WatchSource:0}: Error finding container 75127ff174f070fd113f3278cd62b305707813bb4b36917d47f1d2e77c954dcc: Status 404 returned error can't find the container with id 75127ff174f070fd113f3278cd62b305707813bb4b36917d47f1d2e77c954dcc
Sep 08 13:30:29.041096 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.041053    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5lhct_1149f2f0-504e-4f23-9a7e-893aa93405d7/serve-healthcheck-canary/0.log"
Sep 08 13:30:29.155276 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.155192    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:29.155276 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.155249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.155294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.155316    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155344    2583 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155362    2583 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b8d5d74d6-xh8f4: secret "image-registry-tls" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155409    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155417    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155419    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls podName:3c611e37-7835-408e-a108-695d1e8b978e nodeName:}" failed. No retries permitted until 2026-09-08 13:30:37.15540117 +0000 UTC m=+123.373317472 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls") pod "image-registry-5b8d5d74d6-xh8f4" (UID: "3c611e37-7835-408e-a108-695d1e8b978e") : secret "image-registry-tls" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155413    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155481    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:33.155464888 +0000 UTC m=+119.373381196 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:29.155498 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155506    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:37.155489862 +0000 UTC m=+123.373406183 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:29.155862 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:29.155525    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:33.155516305 +0000 UTC m=+119.373432612 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:29.256574 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.256517    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:29.313019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.312988    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:29.658421 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.658385    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-7r842" event={"ID":"0d2e6582-d29c-421b-bf87-200717f8dd40","Type":"ContainerStarted","Data":"e7bb5c823c3af039eb74f2d30b067d4c52a3bfa2d2b15824d8dbf175a375bfd3"}
Sep 08 13:30:29.658604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.658442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-75d99c484-7r842" event={"ID":"0d2e6582-d29c-421b-bf87-200717f8dd40","Type":"ContainerStarted","Data":"75127ff174f070fd113f3278cd62b305707813bb4b36917d47f1d2e77c954dcc"}
Sep 08 13:30:29.677480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:29.677427    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-75d99c484-7r842" podStartSLOduration=1.677411764 podStartE2EDuration="1.677411764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:29.676730148 +0000 UTC m=+115.894646491" watchObservedRunningTime="2026-09-08 13:30:29.677411764 +0000 UTC m=+115.895328090"
Sep 08 13:30:30.040617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:30.040572    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:30.151227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:30.151189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:30.246865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:30.246828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:30.346883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:30.346786    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:30.411447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:30.411409    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:31.340034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:31.339996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:31.536934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:31.536898    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:31.729913 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:31.729824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:32.021113 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:32.019317    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:32.135353 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:32.135319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:32.534071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:32.534028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:33.188943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.188902    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:33.189438 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.188986    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:33.189438 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:33.189047    2583 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 08 13:30:33.189438 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:33.189094    2583 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:33.189438 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:33.189131    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:41.189111246 +0000 UTC m=+127.407027572 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : configmap "kube-rbac-proxy" not found
Sep 08 13:30:33.189438 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:33.189158    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls podName:d4635088-a268-4e79-8ae3-ebaf77e812e1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:41.189141687 +0000 UTC m=+127.407057989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls") pod "insights-runtime-extractor-wsjt9" (UID: "d4635088-a268-4e79-8ae3-ebaf77e812e1") : secret "insights-runtime-extractor-tls" not found
Sep 08 13:30:33.339227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.339184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:33.536483 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.536452    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:33.845340 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.845249    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:33.945623 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:33.945581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:34.001481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:34.001431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:34.134157 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:34.134063    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:34.536230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:34.536191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:35.342845 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:35.342808    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:35.533527 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:35.533477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:35.943493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:35.943457    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:36.045145 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:36.045111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:36.144053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:36.144016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:36.541458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:36.541423    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:37.212280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.212247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:37.222214 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.222186    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:37.222294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.222243    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:37.222347 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:37.222331    2583 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 08 13:30:37.222410 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:30:37.222399    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert podName:308c88fa-8f8c-4646-8c3a-8bc55633bbc1 nodeName:}" failed. No retries permitted until 2026-09-08 13:30:53.222381707 +0000 UTC m=+139.440298009 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert") pod "networking-console-plugin-79977f9844-g5c67" (UID: "308c88fa-8f8c-4646-8c3a-8bc55633bbc1") : secret "networking-console-plugin-cert" not found
Sep 08 13:30:37.224847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.224824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"image-registry-5b8d5d74d6-xh8f4\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") " pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:37.354363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.354330    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9cf6z\""
Sep 08 13:30:37.362190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.362168    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:37.543810 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.543763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:37.638680 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.638647    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:30:37.641923 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:37.641891    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c611e37_7835_408e_a108_695d1e8b978e.slice/crio-c9ebb1edd10a8999bf8d34cbf64694d607297097603bf47e804175cefbd4e0b4 WatchSource:0}: Error finding container c9ebb1edd10a8999bf8d34cbf64694d607297097603bf47e804175cefbd4e0b4: Status 404 returned error can't find the container with id c9ebb1edd10a8999bf8d34cbf64694d607297097603bf47e804175cefbd4e0b4
Sep 08 13:30:37.687663 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.687632    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" event={"ID":"3c611e37-7835-408e-a108-695d1e8b978e","Type":"ContainerStarted","Data":"c9ebb1edd10a8999bf8d34cbf64694d607297097603bf47e804175cefbd4e0b4"}
Sep 08 13:30:37.934480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:37.934378    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:38.131969 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.131933    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:38.240662 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.240577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:38.534015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.533981    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:38.692307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.692270    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" event={"ID":"3c611e37-7835-408e-a108-695d1e8b978e","Type":"ContainerStarted","Data":"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"}
Sep 08 13:30:38.692492 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.692331    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:30:38.713044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.712991    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" podStartSLOduration=17.712976411 podStartE2EDuration="17.712976411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:38.711872485 +0000 UTC m=+124.929788812" watchObservedRunningTime="2026-09-08 13:30:38.712976411 +0000 UTC m=+124.930892735"
Sep 08 13:30:38.719640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:38.716762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:39.335144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:39.335104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:39.537539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:39.537487    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:39.934006 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:39.933968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:41.256503 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.256446    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:41.256503 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.256521    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:41.257033 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.257013    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d4635088-a268-4e79-8ae3-ebaf77e812e1-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:41.258947 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.258927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d4635088-a268-4e79-8ae3-ebaf77e812e1-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-wsjt9\" (UID: \"d4635088-a268-4e79-8ae3-ebaf77e812e1\") " pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:41.402619 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.402582    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j78zr\""
Sep 08 13:30:41.410567 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.410527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-wsjt9"
Sep 08 13:30:41.679285 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.679242    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-wsjt9"]
Sep 08 13:30:41.683136 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:41.683108    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4635088_a268_4e79_8ae3_ebaf77e812e1.slice/crio-dbebc65a8b6c3db9361e1d4bb39bcc52a2c657a01b56af387070d3553a6a9d40 WatchSource:0}: Error finding container dbebc65a8b6c3db9361e1d4bb39bcc52a2c657a01b56af387070d3553a6a9d40: Status 404 returned error can't find the container with id dbebc65a8b6c3db9361e1d4bb39bcc52a2c657a01b56af387070d3553a6a9d40
Sep 08 13:30:41.703200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:41.703173    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsjt9" event={"ID":"d4635088-a268-4e79-8ae3-ebaf77e812e1","Type":"ContainerStarted","Data":"dbebc65a8b6c3db9361e1d4bb39bcc52a2c657a01b56af387070d3553a6a9d40"}
Sep 08 13:30:42.293878 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:42.293850    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:42.344755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:42.344709    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:42.644748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:42.644640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:42.700393 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:42.700361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:42.708496 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:42.708470    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsjt9" event={"ID":"d4635088-a268-4e79-8ae3-ebaf77e812e1","Type":"ContainerStarted","Data":"88e1b904c03904d20920faaec6c23948b3f9196df7f33bf7ca47dc0766d9c5ca"}
Sep 08 13:30:43.072209 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.072168    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:30:43.074737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.074714    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b10de668-df4d-40b5-9e19-255843710763-metrics-certs\") pod \"network-metrics-daemon-msdpw\" (UID: \"b10de668-df4d-40b5-9e19-255843710763\") " pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:30:43.339846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.339759    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-56h6n\""
Sep 08 13:30:43.348421 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.348394    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-msdpw"
Sep 08 13:30:43.615745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.615652    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-msdpw"]
Sep 08 13:30:43.619941 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:43.619905    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb10de668_df4d_40b5_9e19_255843710763.slice/crio-07a0350f8ef44ea950adab326bca7b0c950c806d47de1ea063828e28d80caa30 WatchSource:0}: Error finding container 07a0350f8ef44ea950adab326bca7b0c950c806d47de1ea063828e28d80caa30: Status 404 returned error can't find the container with id 07a0350f8ef44ea950adab326bca7b0c950c806d47de1ea063828e28d80caa30
Sep 08 13:30:43.714288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.714251    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsjt9" event={"ID":"d4635088-a268-4e79-8ae3-ebaf77e812e1","Type":"ContainerStarted","Data":"0702e213de98a1a3b9dbb3cd679ea41ec88d0327d417a9886a76bec6d24a49f2"}
Sep 08 13:30:43.715605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:43.715573    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msdpw" event={"ID":"b10de668-df4d-40b5-9e19-255843710763","Type":"ContainerStarted","Data":"07a0350f8ef44ea950adab326bca7b0c950c806d47de1ea063828e28d80caa30"}
Sep 08 13:30:44.720688 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:44.720594    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-wsjt9" event={"ID":"d4635088-a268-4e79-8ae3-ebaf77e812e1","Type":"ContainerStarted","Data":"62ab3a53897351f93e6eae98b4e0ad8da642909b346deb5f43497370e9ee79eb"}
Sep 08 13:30:44.738208 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:44.738155    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-wsjt9" podStartSLOduration=17.253255058 podStartE2EDuration="19.738134551s" totalImagesPullingTime="2.484879493s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 13:30:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:44.73631002 +0000 UTC m=+130.954226366" watchObservedRunningTime="2026-09-08 13:30:44.738134551 +0000 UTC m=+130.956050875"
Sep 08 13:30:46.728722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:46.728679    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msdpw" event={"ID":"b10de668-df4d-40b5-9e19-255843710763","Type":"ContainerStarted","Data":"1406146b7886121daf3108e85b6330ede3b5190e38fc07ca49fbab0823d49246"}
Sep 08 13:30:46.729067 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:46.728781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-msdpw" event={"ID":"b10de668-df4d-40b5-9e19-255843710763","Type":"ContainerStarted","Data":"bdabb2cbd4ff1c24be07260540c2dd80155b2352d3c8da9d136919d4142109fb"}
Sep 08 13:30:46.744143 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:46.744096    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-msdpw" podStartSLOduration=130.191402271 podStartE2EDuration="2m12.744081811s" totalImagesPullingTime="2.55267954s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:28:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:46.743291718 +0000 UTC m=+132.961208044" watchObservedRunningTime="2026-09-08 13:30:46.744081811 +0000 UTC m=+132.961998136"
Sep 08 13:30:47.315883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:47.315845    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:47.422118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:47.422084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:47.508943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:47.508911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:48.225175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:48.225121    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:49.069726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.069689    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:30:49.091833 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.091801    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-74b8f69c69-f4p98"]
Sep 08 13:30:49.094970 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.094954    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.106963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.106940    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b8f69c69-f4p98"]
Sep 08 13:30:49.136199 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.136164    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:49.221060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221018    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-bound-sa-token\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221087    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-image-registry-private-configuration\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221127    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-certificates\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dce2af13-dcad-4d45-bef6-78745a498bb3-ca-trust-extracted\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trw8p\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-kube-api-access-trw8p\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221231    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-tls\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221268    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-trusted-ca\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.221415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.221285    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-installation-pull-secrets\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.234129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.234100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:49.322225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322131    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dce2af13-dcad-4d45-bef6-78745a498bb3-ca-trust-extracted\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trw8p\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-kube-api-access-trw8p\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322229    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-tls\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322269    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-trusted-ca\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322294    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-installation-pull-secrets\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322345    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-bound-sa-token\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-image-registry-private-configuration\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322437    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-certificates\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.322719 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.322601    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/dce2af13-dcad-4d45-bef6-78745a498bb3-ca-trust-extracted\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.323331 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.323302    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-trusted-ca\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.323534 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.323509    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-certificates\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.324941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.324912    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-registry-tls\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.325047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.325027    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-installation-pull-secrets\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.325194 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.325173    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/dce2af13-dcad-4d45-bef6-78745a498bb3-image-registry-private-configuration\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.332616 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.332591    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trw8p\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-kube-api-access-trw8p\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.332741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.332725    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce2af13-dcad-4d45-bef6-78745a498bb3-bound-sa-token\") pod \"image-registry-74b8f69c69-f4p98\" (UID: \"dce2af13-dcad-4d45-bef6-78745a498bb3\") " pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.403938 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.403899    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:49.679351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.679249    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-74b8f69c69-f4p98"]
Sep 08 13:30:49.681916 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:49.681891    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddce2af13_dcad_4d45_bef6_78745a498bb3.slice/crio-e304c5594737fb387061280675d6a67342fa6756a35d37f800f110d68a57c1e6 WatchSource:0}: Error finding container e304c5594737fb387061280675d6a67342fa6756a35d37f800f110d68a57c1e6: Status 404 returned error can't find the container with id e304c5594737fb387061280675d6a67342fa6756a35d37f800f110d68a57c1e6
Sep 08 13:30:49.739489 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:49.739462    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98" event={"ID":"dce2af13-dcad-4d45-bef6-78745a498bb3","Type":"ContainerStarted","Data":"e304c5594737fb387061280675d6a67342fa6756a35d37f800f110d68a57c1e6"}
Sep 08 13:30:50.744458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:50.744423    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98" event={"ID":"dce2af13-dcad-4d45-bef6-78745a498bb3","Type":"ContainerStarted","Data":"8aadb516f0c835453a09b4c6b65f92ccfddb4af63a4d449cbf16721005fa69e8"}
Sep 08 13:30:50.744881 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:50.744577    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:30:50.764494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:50.764439    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98" podStartSLOduration=1.764410775 podStartE2EDuration="1.764410775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:50.764141479 +0000 UTC m=+136.982057815" watchObservedRunningTime="2026-09-08 13:30:50.764410775 +0000 UTC m=+136.982327103"
Sep 08 13:30:50.836462 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:50.836431    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:50.897182 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:50.897135    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:53.251396 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:53.251328    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:53.253911 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:53.253887    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/308c88fa-8f8c-4646-8c3a-8bc55633bbc1-networking-console-plugin-cert\") pod \"networking-console-plugin-79977f9844-g5c67\" (UID: \"308c88fa-8f8c-4646-8c3a-8bc55633bbc1\") " pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:53.532762 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:53.532736    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jxxjh\""
Sep 08 13:30:53.540859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:53.540839    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67"
Sep 08 13:30:53.823351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:53.823274    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-79977f9844-g5c67"]
Sep 08 13:30:53.826180 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:30:53.826137    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod308c88fa_8f8c_4646_8c3a_8bc55633bbc1.slice/crio-a6a71150952e752a54b8e8e49b25edd408c10eb147e61721233106ba158e00bb WatchSource:0}: Error finding container a6a71150952e752a54b8e8e49b25edd408c10eb147e61721233106ba158e00bb: Status 404 returned error can't find the container with id a6a71150952e752a54b8e8e49b25edd408c10eb147e61721233106ba158e00bb
Sep 08 13:30:54.759541 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:54.759509    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67" event={"ID":"308c88fa-8f8c-4646-8c3a-8bc55633bbc1","Type":"ContainerStarted","Data":"a6a71150952e752a54b8e8e49b25edd408c10eb147e61721233106ba158e00bb"}
Sep 08 13:30:55.518319 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.518283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:55.586891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.586800    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:55.677244 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.677207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:55.764235 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.764200    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67" event={"ID":"308c88fa-8f8c-4646-8c3a-8bc55633bbc1","Type":"ContainerStarted","Data":"6a1b172c498b383fa6c5206bb9089f6c4ff0fc10fd16bebe8b7b1967be6db03b"}
Sep 08 13:30:55.780917 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.780867    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-79977f9844-g5c67" podStartSLOduration=33.326052018 podStartE2EDuration="34.780850943s" totalImagesPullingTime="1.454798925s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:30:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:30:55.780197178 +0000 UTC m=+141.998113533" watchObservedRunningTime="2026-09-08 13:30:55.780850943 +0000 UTC m=+141.998767268"
Sep 08 13:30:55.873241 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.873150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:55.974431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:55.974397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:30:59.075518 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:59.075483    2583 patch_prober.go:28] interesting pod/image-registry-5b8d5d74d6-xh8f4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 13:30:59.075909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:59.075535    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:30:59.640586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:30:59.640527    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:00.652066 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.652012    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:00.778943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.778905    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:00.903259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.903176    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"]
Sep 08 13:31:00.906269 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.906246    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.908642 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.908617    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 08 13:31:00.908763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.908681    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 08 13:31:00.909471 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.909456    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 08 13:31:00.909614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.909596    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 08 13:31:00.909684 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.909507    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 08 13:31:00.909737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.909508    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 08 13:31:00.909737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.909459    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-t4nqd\""
Sep 08 13:31:00.913680 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913655    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.913803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913719    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.913864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913804    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc15e85c-a68f-4adc-8940-2f4ee46da50c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.913864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913845    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c49tk\" (UniqueName: \"kubernetes.io/projected/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-api-access-c49tk\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.913970 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913881    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.913970 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.913909    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:00.915925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.915748    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-z7rp4"]
Sep 08 13:31:00.918366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.918349    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:00.919451 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.919428    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"]
Sep 08 13:31:00.920771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.920752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 08 13:31:00.920869 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.920752    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 08 13:31:00.921045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.921026    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-lm4lr\""
Sep 08 13:31:00.921234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:00.921209    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 08 13:31:01.015215 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015160    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.015215 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015224    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.015468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015264    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-textfile\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015294    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-wtmp\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015333    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.015468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015423    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-root\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.015693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-tls\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015590    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nzm74\" (UniqueName: \"kubernetes.io/projected/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-kube-api-access-nzm74\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015620    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-metrics-client-ca\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015671    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-sys\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.015929 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:31:01.015838    2583 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 08 13:31:01.015929 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:31:01.015902    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls podName:fc15e85c-a68f-4adc-8940-2f4ee46da50c nodeName:}" failed. No retries permitted until 2026-09-08 13:31:01.515882587 +0000 UTC m=+147.733798911 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls") pod "kube-state-metrics-74cb57df7-swj4m" (UID: "fc15e85c-a68f-4adc-8940-2f4ee46da50c") : secret "kube-state-metrics-tls" not found
Sep 08 13:31:01.016128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015941    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc15e85c-a68f-4adc-8940-2f4ee46da50c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.016128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.015980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c49tk\" (UniqueName: \"kubernetes.io/projected/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-api-access-c49tk\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.016128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.016007    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.016283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.016262    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.016340 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.016289    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/fc15e85c-a68f-4adc-8940-2f4ee46da50c-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.018001 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.017978    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.026778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.026754    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c49tk\" (UniqueName: \"kubernetes.io/projected/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-api-access-c49tk\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.116714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116668    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-textfile\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116710    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-wtmp\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116751    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-root\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116776    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-tls\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116791    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nzm74\" (UniqueName: \"kubernetes.io/projected/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-kube-api-access-nzm74\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-metrics-client-ca\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116852    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.116977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.116864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-root\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.117330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.117006    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-wtmp\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.117330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.117092    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-textfile\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.117330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.117214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-sys\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.117330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.117291    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-sys\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.117781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.117758    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-accelerators-collector-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.118122 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.118103    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-metrics-client-ca\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.119277 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.119255    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.119362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.119345    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-node-exporter-tls\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.125481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.125421    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nzm74\" (UniqueName: \"kubernetes.io/projected/5da9d9e4-aaee-462b-be6d-5fdcf2703eef-kube-api-access-nzm74\") pod \"node-exporter-z7rp4\" (UID: \"5da9d9e4-aaee-462b-be6d-5fdcf2703eef\") " pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.228703 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.228609    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-z7rp4"
Sep 08 13:31:01.520432 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.520396    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.522830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.522807    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/fc15e85c-a68f-4adc-8940-2f4ee46da50c-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-swj4m\" (UID: \"fc15e85c-a68f-4adc-8940-2f4ee46da50c\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.789003 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.788910    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7rp4" event={"ID":"5da9d9e4-aaee-462b-be6d-5fdcf2703eef","Type":"ContainerStarted","Data":"45d6d1c328ce6f36c4287275d22cc0673bab8e59bc6a862bcfaeca08f26b7c5c"}
Sep 08 13:31:01.816137 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.816106    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"
Sep 08 13:31:01.952288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.952253    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:31:01.955251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.955229    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:01.957366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957331    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 13:31:01.957491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957391    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 13:31:01.957491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q57nm\""
Sep 08 13:31:01.957491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957469    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 13:31:01.957668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957328    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 13:31:01.957668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957625    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 13:31:01.957668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957630    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 13:31:01.957811 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957755    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 13:31:01.957811 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957759    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 13:31:01.957910 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.957825    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 13:31:01.969736 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:01.969706    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:31:02.023924 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.023888    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.023924 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.023932    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.023964    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024010    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024031    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024101    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9bxp\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024119    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024171    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024185    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024203    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024218    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.024385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.024323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.088104 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.088077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-swj4m"]
Sep 08 13:31:02.090681 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:02.090650    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc15e85c_a68f_4adc_8940_2f4ee46da50c.slice/crio-5afdb9f800a96c77653a921bda4e62c71047954ee722664ce17ac83ada29a70d WatchSource:0}: Error finding container 5afdb9f800a96c77653a921bda4e62c71047954ee722664ce17ac83ada29a70d: Status 404 returned error can't find the container with id 5afdb9f800a96c77653a921bda4e62c71047954ee722664ce17ac83ada29a70d
Sep 08 13:31:02.125456 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125426    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125469    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125516    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125537    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125574    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125620    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.125665 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:31:02.125647    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle podName:8e5e1176-e7d9-48ba-864b-e495ff1c006d nodeName:}" failed. No retries permitted until 2026-09-08 13:31:02.62562092 +0000 UTC m=+148.843537242 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d") : configmap references non-existent config key: ca-bundle.crt
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125691    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9bxp\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125725    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125768    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125793    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125825    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.126047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.125851    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.127017 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.126987    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.127917 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.127545    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.128837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.128809    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.128927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.128856    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.129336 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.129309    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.129422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.129336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.129688 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.129669    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.129816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.129796    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.130788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.130766    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.131317 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.131284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.131817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.131798    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.136592 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.136572    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9bxp\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.589718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.589678    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:02.629920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.629881    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.630607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.630589    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:02.690875 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.690830    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:02.793445 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.793407    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m" event={"ID":"fc15e85c-a68f-4adc-8940-2f4ee46da50c","Type":"ContainerStarted","Data":"5afdb9f800a96c77653a921bda4e62c71047954ee722664ce17ac83ada29a70d"}
Sep 08 13:31:02.865004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:02.864921    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:31:03.145942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.145894    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:31:03.147900 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:03.147872    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e5e1176_e7d9_48ba_864b_e495ff1c006d.slice/crio-ed608ca54be25df136cb02c2885daaccae54f6287d0e205261da5e0b56510fa9 WatchSource:0}: Error finding container ed608ca54be25df136cb02c2885daaccae54f6287d0e205261da5e0b56510fa9: Status 404 returned error can't find the container with id ed608ca54be25df136cb02c2885daaccae54f6287d0e205261da5e0b56510fa9
Sep 08 13:31:03.799103 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.799064    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"ed608ca54be25df136cb02c2885daaccae54f6287d0e205261da5e0b56510fa9"}
Sep 08 13:31:03.829745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.829712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:03.879384 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.879357    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"]
Sep 08 13:31:03.882546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.882529    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.884871 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.884847    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 08 13:31:03.884974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.884848    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 08 13:31:03.884974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.884919    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-6hhl7\""
Sep 08 13:31:03.885100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.885029    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 08 13:31:03.885100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.885034    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2ic3lovbsvs3r\""
Sep 08 13:31:03.885100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.885039    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 08 13:31:03.885247 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.885238    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 08 13:31:03.894667 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.894641    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"]
Sep 08 13:31:03.901258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.901225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:03.941684 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941653    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1305f4-62ba-43e0-86d4-a940c40610e0-metrics-client-ca\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.941864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941702    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.941864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941740    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-grpc-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.941864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941795    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfvvv\" (UniqueName: \"kubernetes.io/projected/4b1305f4-62ba-43e0-86d4-a940c40610e0-kube-api-access-kfvvv\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.941864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941829    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.941864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.942119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:03.942119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:03.941939    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.042977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.042936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1305f4-62ba-43e0-86d4-a940c40610e0-metrics-client-ca\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043003    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-grpc-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfvvv\" (UniqueName: \"kubernetes.io/projected/4b1305f4-62ba-43e0-86d4-a940c40610e0-kube-api-access-kfvvv\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043090    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043157    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.043481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043214    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.044159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.043737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4b1305f4-62ba-43e0-86d4-a940c40610e0-metrics-client-ca\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.046416 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.046386    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-grpc-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.046506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.046416    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.046882 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.046864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-tls\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.046936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.046874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.047001 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.046954    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.047433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.047412    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4b1305f4-62ba-43e0-86d4-a940c40610e0-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.051124 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.051073    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfvvv\" (UniqueName: \"kubernetes.io/projected/4b1305f4-62ba-43e0-86d4-a940c40610e0-kube-api-access-kfvvv\") pod \"thanos-querier-6c5b7746c4-rx96s\" (UID: \"4b1305f4-62ba-43e0-86d4-a940c40610e0\") " pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.158954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.158913    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:04.191874 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.191820    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:04.269063 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.269028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:04.481431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.481340    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"]
Sep 08 13:31:04.639015 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:04.638984    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b1305f4_62ba_43e0_86d4_a940c40610e0.slice/crio-7e4ee008e52773a9eb6d8335c121540de57d2c9e0cf6ab6b15f8b9d831ba33b7 WatchSource:0}: Error finding container 7e4ee008e52773a9eb6d8335c121540de57d2c9e0cf6ab6b15f8b9d831ba33b7: Status 404 returned error can't find the container with id 7e4ee008e52773a9eb6d8335c121540de57d2c9e0cf6ab6b15f8b9d831ba33b7
Sep 08 13:31:04.804211 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.804171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"7e4ee008e52773a9eb6d8335c121540de57d2c9e0cf6ab6b15f8b9d831ba33b7"}
Sep 08 13:31:04.806064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.806032    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71" exitCode=0
Sep 08 13:31:04.806190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:04.806077    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"}
Sep 08 13:31:05.675196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.675116    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"]
Sep 08 13:31:05.677704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.677682    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.680007 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.679906    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 08 13:31:05.680122 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.680062    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4z7sj\""
Sep 08 13:31:05.681842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.681788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:05.687444 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.686725    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"]
Sep 08 13:31:05.757764 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.757728    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b08705b1-29a2-47c9-af8f-b8ead9b25831-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.757964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.757940    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq5th\" (UniqueName: \"kubernetes.io/projected/b08705b1-29a2-47c9-af8f-b8ead9b25831-kube-api-access-vq5th\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.812035 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.811993    2583 generic.go:356] "Generic (PLEG): container finished" podID="5da9d9e4-aaee-462b-be6d-5fdcf2703eef" containerID="2e5c9ce023e6f3f7584c9fb5450f01bb3434d4c1b8a5e790ee3eaedc5b5d55d7" exitCode=0
Sep 08 13:31:05.812456 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.812056    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7rp4" event={"ID":"5da9d9e4-aaee-462b-be6d-5fdcf2703eef","Type":"ContainerDied","Data":"2e5c9ce023e6f3f7584c9fb5450f01bb3434d4c1b8a5e790ee3eaedc5b5d55d7"}
Sep 08 13:31:05.858842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.858807    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq5th\" (UniqueName: \"kubernetes.io/projected/b08705b1-29a2-47c9-af8f-b8ead9b25831-kube-api-access-vq5th\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.858842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.858854    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b08705b1-29a2-47c9-af8f-b8ead9b25831-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.862375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.862327    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b08705b1-29a2-47c9-af8f-b8ead9b25831-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.870625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.870587    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq5th\" (UniqueName: \"kubernetes.io/projected/b08705b1-29a2-47c9-af8f-b8ead9b25831-kube-api-access-vq5th\") pod \"monitoring-plugin-85bbbbbf4-tzcwg\" (UID: \"b08705b1-29a2-47c9-af8f-b8ead9b25831\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:05.887106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.887082    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:05.991206 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:05.991162    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:06.623846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.623770    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"]
Sep 08 13:31:06.635351 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:06.635316    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb08705b1_29a2_47c9_af8f_b8ead9b25831.slice/crio-12424795ea7fe940483e4ec2ac8edd52960dc002907caf772a1234fd81ae54da WatchSource:0}: Error finding container 12424795ea7fe940483e4ec2ac8edd52960dc002907caf772a1234fd81ae54da: Status 404 returned error can't find the container with id 12424795ea7fe940483e4ec2ac8edd52960dc002907caf772a1234fd81ae54da
Sep 08 13:31:06.818289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.818244    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg" event={"ID":"b08705b1-29a2-47c9-af8f-b8ead9b25831","Type":"ContainerStarted","Data":"12424795ea7fe940483e4ec2ac8edd52960dc002907caf772a1234fd81ae54da"}
Sep 08 13:31:06.821504 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.821472    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"}
Sep 08 13:31:06.821674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.821575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"}
Sep 08 13:31:06.821674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.821624    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"}
Sep 08 13:31:06.821674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.821666    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"}
Sep 08 13:31:06.823981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.823958    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7rp4" event={"ID":"5da9d9e4-aaee-462b-be6d-5fdcf2703eef","Type":"ContainerStarted","Data":"665920ff7f3a8a3e07c44820e39b15228957bb56ff97bc64d1a3c5f9a2cd2d98"}
Sep 08 13:31:06.824080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.824006    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-z7rp4" event={"ID":"5da9d9e4-aaee-462b-be6d-5fdcf2703eef","Type":"ContainerStarted","Data":"0e59dddcc2ece6d6c30f16b7688a232fdbcb7070ed961f903603ef0a7818854c"}
Sep 08 13:31:06.840330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:06.840264    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-z7rp4" podStartSLOduration=2.805768998 podStartE2EDuration="6.840244227s" totalImagesPullingTime="3.957879723s" totalInitContainerRuntime="76.595506ms" isStatefulPod=true podCreationTimestamp="2026-09-08 13:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:06.839822742 +0000 UTC m=+153.057739085" watchObservedRunningTime="2026-09-08 13:31:06.840244227 +0000 UTC m=+153.058160553"
Sep 08 13:31:07.049341 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.049307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:07.833874 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.833822    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"}
Sep 08 13:31:07.838362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.838330    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m" event={"ID":"fc15e85c-a68f-4adc-8940-2f4ee46da50c","Type":"ContainerStarted","Data":"b8c9d840ce9324b08cb1fc907bd5b6350ab13114a6e87fb42af73aeb04b082b1"}
Sep 08 13:31:07.838525 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.838395    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m" event={"ID":"fc15e85c-a68f-4adc-8940-2f4ee46da50c","Type":"ContainerStarted","Data":"cb7d91545192bcb02ac4a938572300dcc5af9177adaaf04051565a08d15e4bc5"}
Sep 08 13:31:07.838525 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.838423    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m" event={"ID":"fc15e85c-a68f-4adc-8940-2f4ee46da50c","Type":"ContainerStarted","Data":"0de0f7673516b3c1edb71fb365b3f9de3e0e861bf09aaa864d93f8a169643eb2"}
Sep 08 13:31:07.847227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.847192    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"09ddbac5239b6da2f5502b02bf31cacba4f2ba69a07c874ad97a799f59e41703"}
Sep 08 13:31:07.847383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.847273    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"bbae33d65b624374428cc4bf492fbd56353a1fceb375bcc266551faeb7656f99"}
Sep 08 13:31:07.847383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:07.847312    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"5306e47729390a4d12ad65c6d0e6bc28cc8b4718d4b5e88c4ee0e6e2b80ecd95"}
Sep 08 13:31:08.136817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.136739    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:08.252180 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.252139    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:08.853404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.853306    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"549cead1778bdeb219b4705f3f66e484e78d23c8b0cc67eff2260759d6b8b60e"}
Sep 08 13:31:08.853836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.853415    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"5c79855f477b9a2b84beab1a7d70a9dae84c59790da7255a010c4582a2961d66"}
Sep 08 13:31:08.853836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.853462    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:08.853836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.853495    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" event={"ID":"4b1305f4-62ba-43e0-86d4-a940c40610e0","Type":"ContainerStarted","Data":"90a9a4484925cf1939ce026297e7d80313a05393cb5cf52e46f180eb7e99943b"}
Sep 08 13:31:08.854949 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.854924    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg" event={"ID":"b08705b1-29a2-47c9-af8f-b8ead9b25831","Type":"ContainerStarted","Data":"d673f05cade2548df215468bd57685b7a911160b97ecc1f20344f43febe1478f"}
Sep 08 13:31:08.855106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.855082    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:08.858676 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.858650    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerStarted","Data":"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"}
Sep 08 13:31:08.860788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.860768    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg"
Sep 08 13:31:08.874745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.874697    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-swj4m" podStartSLOduration=3.42496323 podStartE2EDuration="8.874677923s" totalImagesPullingTime="5.449714693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:07.857668751 +0000 UTC m=+154.075585096" watchObservedRunningTime="2026-09-08 13:31:08.874677923 +0000 UTC m=+155.092594249"
Sep 08 13:31:08.875514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.875485    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s" podStartSLOduration=2.168014529 podStartE2EDuration="5.875478173s" totalImagesPullingTime="3.707463644s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:08.873035079 +0000 UTC m=+155.090951427" watchObservedRunningTime="2026-09-08 13:31:08.875478173 +0000 UTC m=+155.093394497"
Sep 08 13:31:08.897060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.897016    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.825571431 podStartE2EDuration="7.897001798s" totalImagesPullingTime="4.05447255s" totalInitContainerRuntime="16.957817ms" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:01 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:08.895317995 +0000 UTC m=+155.113234358" watchObservedRunningTime="2026-09-08 13:31:08.897001798 +0000 UTC m=+155.114918159"
Sep 08 13:31:08.912358 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:08.912305    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-tzcwg" podStartSLOduration=2.2971025640000002 podStartE2EDuration="3.91228754s" totalImagesPullingTime="1.615184976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:08.911535117 +0000 UTC m=+155.129451482" watchObservedRunningTime="2026-09-08 13:31:08.91228754 +0000 UTC m=+155.130203865"
Sep 08 13:31:09.074381 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:09.074345    2583 patch_prober.go:28] interesting pod/image-registry-5b8d5d74d6-xh8f4 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 13:31:09.074528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:09.074399    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:31:09.409015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:09.408982    2583 patch_prober.go:28] interesting pod/image-registry-74b8f69c69-f4p98 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 08 13:31:09.409176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:09.409032    2583 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98" podUID="dce2af13-dcad-4d45-bef6-78745a498bb3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:31:10.159782 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:31:10.159735    2583 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-g2j96" podUID="0e7a9712-c9fa-4a81-8bee-304a282f05b6"
Sep 08 13:31:10.867546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:10.867516    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:11.753588 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:11.753538    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-74b8f69c69-f4p98"
Sep 08 13:31:11.882664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:11.882629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:11.986343 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:11.986305    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:12.083527 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:12.083488    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:14.089469 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.089423    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry" containerID="cri-o://b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27" gracePeriod=30
Sep 08 13:31:14.328328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.328304    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:31:14.436297 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436211    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436297 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436259    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436297 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436285    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436313    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436331    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82zls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436359    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436375    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436434    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted\") pod \"3c611e37-7835-408e-a108-695d1e8b978e\" (UID: \"3c611e37-7835-408e-a108-695d1e8b978e\") "
Sep 08 13:31:14.436911 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436884    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:31:14.436985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.436911    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:31:14.438814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.438781    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:31:14.438814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.438788    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:31:14.439250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.439216    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:31:14.439368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.439343    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls" (OuterVolumeSpecName: "kube-api-access-82zls") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "kube-api-access-82zls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:31:14.439411 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.439356    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:31:14.445028 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.445005    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "3c611e37-7835-408e-a108-695d1e8b978e" (UID: "3c611e37-7835-408e-a108-695d1e8b978e"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:31:14.537160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537117    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-trusted-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537153    2583 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-registry-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537163    2583 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3c611e37-7835-408e-a108-695d1e8b978e-ca-trust-extracted\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537178    2583 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-image-registry-private-configuration\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537188    2583 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3c611e37-7835-408e-a108-695d1e8b978e-registry-certificates\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537196    2583 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3c611e37-7835-408e-a108-695d1e8b978e-installation-pull-secrets\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537204    2583 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-bound-sa-token\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.537413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.537213    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82zls\" (UniqueName: \"kubernetes.io/projected/3c611e37-7835-408e-a108-695d1e8b978e-kube-api-access-82zls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:31:14.870540 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.870513    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6c5b7746c4-rx96s"
Sep 08 13:31:14.885293 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.885260    2583 generic.go:356] "Generic (PLEG): container finished" podID="3c611e37-7835-408e-a108-695d1e8b978e" containerID="b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27" exitCode=0
Sep 08 13:31:14.885410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.885302    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"
Sep 08 13:31:14.885410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.885315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" event={"ID":"3c611e37-7835-408e-a108-695d1e8b978e","Type":"ContainerDied","Data":"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"}
Sep 08 13:31:14.885410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.885359    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b8d5d74d6-xh8f4" event={"ID":"3c611e37-7835-408e-a108-695d1e8b978e","Type":"ContainerDied","Data":"c9ebb1edd10a8999bf8d34cbf64694d607297097603bf47e804175cefbd4e0b4"}
Sep 08 13:31:14.885410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.885387    2583 scope.go:117] "RemoveContainer" containerID="b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"
Sep 08 13:31:14.894515 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.894497    2583 scope.go:117] "RemoveContainer" containerID="b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"
Sep 08 13:31:14.894819 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:31:14.894802    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27\": container with ID starting with b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27 not found: ID does not exist" containerID="b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"
Sep 08 13:31:14.894883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.894826    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27"} err="failed to get container status \"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27\": rpc error: code = NotFound desc = could not find container \"b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27\": container with ID starting with b077107c293ec18cec637f657df34c9a1cfc2ba5d8e07a75280d2e72a923ac27 not found: ID does not exist"
Sep 08 13:31:14.911389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.908920    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:31:14.914599 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:14.914576    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b8d5d74d6-xh8f4"]
Sep 08 13:31:15.040718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.040674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:15.043062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.043032    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0e7a9712-c9fa-4a81-8bee-304a282f05b6-metrics-tls\") pod \"dns-default-g2j96\" (UID: \"0e7a9712-c9fa-4a81-8bee-304a282f05b6\") " pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:15.070046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.070025    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-jwgzv\""
Sep 08 13:31:15.078760 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.078743    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:15.352463 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.352429    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g2j96"]
Sep 08 13:31:15.355217 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:15.355188    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7a9712_c9fa_4a81_8bee_304a282f05b6.slice/crio-9b0d716d6813f4918aa4cfac38dcc027cd7349b633edf0cf2df83c5b1127a4d1 WatchSource:0}: Error finding container 9b0d716d6813f4918aa4cfac38dcc027cd7349b633edf0cf2df83c5b1127a4d1: Status 404 returned error can't find the container with id 9b0d716d6813f4918aa4cfac38dcc027cd7349b633edf0cf2df83c5b1127a4d1
Sep 08 13:31:15.890854 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:15.890815    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2j96" event={"ID":"0e7a9712-c9fa-4a81-8bee-304a282f05b6","Type":"ContainerStarted","Data":"9b0d716d6813f4918aa4cfac38dcc027cd7349b633edf0cf2df83c5b1127a4d1"}
Sep 08 13:31:16.231243 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.231155    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3c611e37-7835-408e-a108-695d1e8b978e" path="/var/lib/kubelet/pods/3c611e37-7835-408e-a108-695d1e8b978e/volumes"
Sep 08 13:31:16.805842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.805808    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-dfvfg"]
Sep 08 13:31:16.806298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.806229    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry"
Sep 08 13:31:16.806298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.806246    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry"
Sep 08 13:31:16.806406 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.806346    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3c611e37-7835-408e-a108-695d1e8b978e" containerName="registry"
Sep 08 13:31:16.809075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.809051    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:16.811477 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.811450    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 08 13:31:16.811477 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.811460    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 08 13:31:16.811686 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.811460    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-wlhbb\""
Sep 08 13:31:16.819756 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.819734    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-dfvfg"]
Sep 08 13:31:16.829152 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.829119    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:16.855141 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.855111    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r96tg\" (UniqueName: \"kubernetes.io/projected/973a6fba-efe8-4903-8e80-db8ce1a831d1-kube-api-access-r96tg\") pod \"downloads-85d98c998f-dfvfg\" (UID: \"973a6fba-efe8-4903-8e80-db8ce1a831d1\") " pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:16.881674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.881641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:16.956487 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.956454    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r96tg\" (UniqueName: \"kubernetes.io/projected/973a6fba-efe8-4903-8e80-db8ce1a831d1-kube-api-access-r96tg\") pod \"downloads-85d98c998f-dfvfg\" (UID: \"973a6fba-efe8-4903-8e80-db8ce1a831d1\") " pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:16.964633 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:16.964594    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r96tg\" (UniqueName: \"kubernetes.io/projected/973a6fba-efe8-4903-8e80-db8ce1a831d1-kube-api-access-r96tg\") pod \"downloads-85d98c998f-dfvfg\" (UID: \"973a6fba-efe8-4903-8e80-db8ce1a831d1\") " pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:17.075233 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.075150    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:17.120673 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.120643    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:17.403814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.403738    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-dfvfg"]
Sep 08 13:31:17.406975 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:17.406947    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod973a6fba_efe8_4903_8e80_db8ce1a831d1.slice/crio-ec1b9bf9dc5f2518da72deb9b2d6ed28e73d4c1690e83815bff94a71bbf5fd08 WatchSource:0}: Error finding container ec1b9bf9dc5f2518da72deb9b2d6ed28e73d4c1690e83815bff94a71bbf5fd08: Status 404 returned error can't find the container with id ec1b9bf9dc5f2518da72deb9b2d6ed28e73d4c1690e83815bff94a71bbf5fd08
Sep 08 13:31:17.448146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.448111    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:17.854860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.854821    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:17.900974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.900940    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2j96" event={"ID":"0e7a9712-c9fa-4a81-8bee-304a282f05b6","Type":"ContainerStarted","Data":"f0e0f3fe46c08c06c4fb1de20ca39fd000eabe639aa5a18c1c923e980192fec3"}
Sep 08 13:31:17.901170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.901005    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g2j96" event={"ID":"0e7a9712-c9fa-4a81-8bee-304a282f05b6","Type":"ContainerStarted","Data":"863512b8a10b430774319874eb094f92fb204f7bcac21d2a3f7e7e0f60122928"}
Sep 08 13:31:17.901170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.901035    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:17.902262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.902233    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-dfvfg" event={"ID":"973a6fba-efe8-4903-8e80-db8ce1a831d1","Type":"ContainerStarted","Data":"ec1b9bf9dc5f2518da72deb9b2d6ed28e73d4c1690e83815bff94a71bbf5fd08"}
Sep 08 13:31:17.917454 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:17.917406    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g2j96" podStartSLOduration=129.238520979 podStartE2EDuration="2m10.917389694s" totalImagesPullingTime="1.678868715s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:29:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:17.916375424 +0000 UTC m=+164.134291759" watchObservedRunningTime="2026-09-08 13:31:17.917389694 +0000 UTC m=+164.135306022"
Sep 08 13:31:18.258190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:18.258084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:21.489241 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:21.489133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:21.546250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:21.546213    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:22.014662 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:22.014618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:22.115762 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:22.115716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:23.206156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.206125    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:31:23.209568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.209527    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.211688 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.211653    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 08 13:31:23.212576 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.212433    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 08 13:31:23.212576 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.212511    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 08 13:31:23.212744 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.212638    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 08 13:31:23.213147 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.212983    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 08 13:31:23.213147 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.213055    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-5kcgz\""
Sep 08 13:31:23.219704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.219678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:31:23.313577 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313520    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.313742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313602    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.313742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313634    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.313742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313658    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bg6f\" (UniqueName: \"kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.313742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313730    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.313958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.313760    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414323    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414392    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414443    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414500    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bg6f\" (UniqueName: \"kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.414866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.414544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.415268 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.415243    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.415404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.415303    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.415630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.415603    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.417301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.417280    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.417411 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.417324    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.424065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.424043    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bg6f\" (UniqueName: \"kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f\") pod \"console-bf5b5d47d-hrp8p\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") " pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.522162 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.522121    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:23.804624 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.804515    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:31:23.808289 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:23.808265    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod556f2462_f3bd_47cc_a247_abb6fdcf14ed.slice/crio-a1fa4e1478615c8886dc39fa98352c34025d6b821aa27153aa445510c025cbb5 WatchSource:0}: Error finding container a1fa4e1478615c8886dc39fa98352c34025d6b821aa27153aa445510c025cbb5: Status 404 returned error can't find the container with id a1fa4e1478615c8886dc39fa98352c34025d6b821aa27153aa445510c025cbb5
Sep 08 13:31:23.864317 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.864284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:23.930116 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:23.930072    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf5b5d47d-hrp8p" event={"ID":"556f2462-f3bd-47cc-a247-abb6fdcf14ed","Type":"ContainerStarted","Data":"a1fa4e1478615c8886dc39fa98352c34025d6b821aa27153aa445510c025cbb5"}
Sep 08 13:31:24.473079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:24.473034    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:24.524362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:24.524312    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:24.888798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:24.888751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:24.940771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:24.940735    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:27.077068 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.076848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:27.133234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.133187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:27.668037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.667897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:27.765988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.765943    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:27.908901 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.908875    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g2j96"
Sep 08 13:31:27.954045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.953957    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf5b5d47d-hrp8p" event={"ID":"556f2462-f3bd-47cc-a247-abb6fdcf14ed","Type":"ContainerStarted","Data":"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"}
Sep 08 13:31:27.973250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:27.973165    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bf5b5d47d-hrp8p" podStartSLOduration=1.483903598 podStartE2EDuration="4.973142099s" totalImagesPullingTime="3.489238501s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:27.972478886 +0000 UTC m=+174.190395330" watchObservedRunningTime="2026-09-08 13:31:27.973142099 +0000 UTC m=+174.191058426"
Sep 08 13:31:29.882748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:29.882698    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:29.982656 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:29.982601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:33.523014 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:33.522973    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:33.523571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:33.523281    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:33.528935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:33.528909    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:33.956703 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:33.956615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:33.988478 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:33.988450    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:31:34.017915 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.017884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:34.081261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.081223    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:31:34.085714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.085693    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.094481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.094459    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 13:31:34.094616 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.094505    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:31:34.210565 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210470    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5p9w\" (UniqueName: \"kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210573    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210656    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210704    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.210935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.210851    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.311949 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.311904    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.311973    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5p9w\" (UniqueName: \"kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312010    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312028    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312061    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312238    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312798    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.312876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312864    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.313064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.312892    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.314984 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.314955    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.315461 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.315440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.319543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.319519    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 08 13:31:34.323239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.323214    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.326455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.326429    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5p9w\" (UniqueName: \"kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w\") pod \"console-68f7fbbb55-swbzd\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") " pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.397189 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.397143    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:34.725694 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.725617    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:31:34.729033 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:31:34.729002    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc32a5f8_f765_465f_a542_fb3803cb603f.slice/crio-7489619cf5135f7f5173fb60bf71baebf10cf377312ec6deb090d3d101aba2ee WatchSource:0}: Error finding container 7489619cf5135f7f5173fb60bf71baebf10cf377312ec6deb090d3d101aba2ee: Status 404 returned error can't find the container with id 7489619cf5135f7f5173fb60bf71baebf10cf377312ec6deb090d3d101aba2ee
Sep 08 13:31:34.991844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.991512    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-dfvfg" event={"ID":"973a6fba-efe8-4903-8e80-db8ce1a831d1","Type":"ContainerStarted","Data":"9d992782d7242fb42f7e3d5a09243bc1580079beae0ac25575921d5bbeca2f80"}
Sep 08 13:31:34.991844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.991613    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:34.993821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.993790    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f7fbbb55-swbzd" event={"ID":"fc32a5f8-f765-465f-a542-fb3803cb603f","Type":"ContainerStarted","Data":"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"}
Sep 08 13:31:34.993954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:34.993841    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f7fbbb55-swbzd" event={"ID":"fc32a5f8-f765-465f-a542-fb3803cb603f","Type":"ContainerStarted","Data":"7489619cf5135f7f5173fb60bf71baebf10cf377312ec6deb090d3d101aba2ee"}
Sep 08 13:31:35.008608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:35.008537    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-dfvfg" podStartSLOduration=1.866761421 podStartE2EDuration="19.008516792s" totalImagesPullingTime="17.141755371s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:35.006708069 +0000 UTC m=+181.224624443" watchObservedRunningTime="2026-09-08 13:31:35.008516792 +0000 UTC m=+181.226433117"
Sep 08 13:31:35.009589 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:35.009539    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-dfvfg"
Sep 08 13:31:35.023495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:35.023433    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68f7fbbb55-swbzd" podStartSLOduration=1.023412524 podStartE2EDuration="1.023412524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:31:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:31:35.021749712 +0000 UTC m=+181.239666063" watchObservedRunningTime="2026-09-08 13:31:35.023412524 +0000 UTC m=+181.241328850"
Sep 08 13:31:37.035633 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:37.035599    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:42.653500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:42.653456    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:42.711225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:42.711187    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:44.398046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:44.398003    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:44.398531 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:44.398081    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:44.403080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:44.403058    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:45.045672 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:45.045643    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:31:45.092469 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:45.092432    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:31:45.497493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:45.497410    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:45.559731 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:45.559707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:46.049715 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:46.049673    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:46.107182 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:46.107151    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:46.294061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:46.294016    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:46.394725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:46.394631    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:47.888680 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:47.888644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:47.988944 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:47.988912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:55.083395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.083360    2583 generic.go:356] "Generic (PLEG): container finished" podID="a9be3d96-349b-465c-af4f-5aeac1612d32" containerID="d7c78423ec03666e40bb1a42fbeb8ba52c8a1324baba3daf3a5b178e3973575f" exitCode=0
Sep 08 13:31:55.083909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.083435    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r" event={"ID":"a9be3d96-349b-465c-af4f-5aeac1612d32","Type":"ContainerDied","Data":"d7c78423ec03666e40bb1a42fbeb8ba52c8a1324baba3daf3a5b178e3973575f"}
Sep 08 13:31:55.083909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.083853    2583 scope.go:117] "RemoveContainer" containerID="d7c78423ec03666e40bb1a42fbeb8ba52c8a1324baba3daf3a5b178e3973575f"
Sep 08 13:31:55.085220 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.085199    2583 generic.go:356] "Generic (PLEG): container finished" podID="bfb83340-d2b2-4688-89c8-5e55a96c0d01" containerID="b1b5ba2b0bded6a61f076f278f6beb857c7eb8f0427a0c58cf9457e170f64af5" exitCode=0
Sep 08 13:31:55.085295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.085235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh" event={"ID":"bfb83340-d2b2-4688-89c8-5e55a96c0d01","Type":"ContainerDied","Data":"b1b5ba2b0bded6a61f076f278f6beb857c7eb8f0427a0c58cf9457e170f64af5"}
Sep 08 13:31:55.085506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:55.085493    2583 scope.go:117] "RemoveContainer" containerID="b1b5ba2b0bded6a61f076f278f6beb857c7eb8f0427a0c58cf9457e170f64af5"
Sep 08 13:31:56.091478 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:56.091446    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-859f94c4b-7zh5r" event={"ID":"a9be3d96-349b-465c-af4f-5aeac1612d32","Type":"ContainerStarted","Data":"af5f19fa7dca1c8480a8b8056f73c2faa839b8a4f5d6e9224300d2a2c72f884e"}
Sep 08 13:31:56.093500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:56.093465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-76bcd6ccf5-zwcmh" event={"ID":"bfb83340-d2b2-4688-89c8-5e55a96c0d01","Type":"ContainerStarted","Data":"fe1e3467265259de98fbfc60d73379979b1a63c1a00dd9a1f2c8c25c15615d90"}
Sep 08 13:31:57.650797 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:57.650748    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:31:57.751453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:31:57.751412    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:10.117354 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.117309    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bf5b5d47d-hrp8p" podUID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" containerName="console" containerID="cri-o://bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee" gracePeriod=15
Sep 08 13:32:10.372644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.372583    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:32:10.533900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.533864    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.533938    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534008    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534027    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534046    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534075    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8bg6f\" (UniqueName: \"kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f\") pod \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\" (UID: \"556f2462-f3bd-47cc-a247-abb6fdcf14ed\") "
Sep 08 13:32:10.534379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534355    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca" (OuterVolumeSpecName: "service-ca") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:10.534707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534677    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:10.534707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.534681    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config" (OuterVolumeSpecName: "console-config") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:10.536435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.536414    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:10.536538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.536470    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f" (OuterVolumeSpecName: "kube-api-access-8bg6f") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "kube-api-access-8bg6f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:32:10.536538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.536498    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "556f2462-f3bd-47cc-a247-abb6fdcf14ed" (UID: "556f2462-f3bd-47cc-a247-abb6fdcf14ed"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635363    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bg6f\" (UniqueName: \"kubernetes.io/projected/556f2462-f3bd-47cc-a247-abb6fdcf14ed-kube-api-access-8bg6f\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635395    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635408    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635420    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635431    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:10.635453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:10.635443    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/556f2462-f3bd-47cc-a247-abb6fdcf14ed-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:11.154303 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.154265    2583 generic.go:356] "Generic (PLEG): container finished" podID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" containerID="bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee" exitCode=0
Sep 08 13:32:11.154776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.154312    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bf5b5d47d-hrp8p"
Sep 08 13:32:11.154776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.154320    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf5b5d47d-hrp8p" event={"ID":"556f2462-f3bd-47cc-a247-abb6fdcf14ed","Type":"ContainerDied","Data":"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"}
Sep 08 13:32:11.154776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.154367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bf5b5d47d-hrp8p" event={"ID":"556f2462-f3bd-47cc-a247-abb6fdcf14ed","Type":"ContainerDied","Data":"a1fa4e1478615c8886dc39fa98352c34025d6b821aa27153aa445510c025cbb5"}
Sep 08 13:32:11.154776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.154388    2583 scope.go:117] "RemoveContainer" containerID="bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"
Sep 08 13:32:11.163049 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.163029    2583 scope.go:117] "RemoveContainer" containerID="bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"
Sep 08 13:32:11.163293 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:11.163272    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee\": container with ID starting with bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee not found: ID does not exist" containerID="bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"
Sep 08 13:32:11.163351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.163304    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee"} err="failed to get container status \"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee\": rpc error: code = NotFound desc = could not find container \"bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee\": container with ID starting with bb653dbc7e2f065437d1b120e8f553f779f52898d12430ffcbb9dcf55a9299ee not found: ID does not exist"
Sep 08 13:32:11.175001 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.174976    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:32:11.178161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:11.178138    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bf5b5d47d-hrp8p"]
Sep 08 13:32:12.229565 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:12.229531    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" path="/var/lib/kubelet/pods/556f2462-f3bd-47cc-a247-abb6fdcf14ed/volumes"
Sep 08 13:32:14.007268 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:14.007240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:14.059011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:14.058977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:20.062118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:20.062084    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:20.162731 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:20.162688    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:21.194649 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.194615    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:21.195054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.194878    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="alertmanager" containerID="cri-o://eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20" gracePeriod=120
Sep 08 13:32:21.195054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.194947    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-metric" containerID="cri-o://f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323" gracePeriod=120
Sep 08 13:32:21.195054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.195004    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="config-reloader" containerID="cri-o://0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd" gracePeriod=120
Sep 08 13:32:21.195054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.194993    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="prom-label-proxy" containerID="cri-o://694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa" gracePeriod=120
Sep 08 13:32:21.195273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.195007    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy" containerID="cri-o://83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5" gracePeriod=120
Sep 08 13:32:21.195657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:21.195249    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-web" containerID="cri-o://143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2" gracePeriod=120
Sep 08 13:32:22.207678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207646    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa" exitCode=0
Sep 08 13:32:22.207678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207673    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5" exitCode=0
Sep 08 13:32:22.207678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207681    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd" exitCode=0
Sep 08 13:32:22.207678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207687    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20" exitCode=0
Sep 08 13:32:22.208184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207716    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"}
Sep 08 13:32:22.208184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207801    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"}
Sep 08 13:32:22.208184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207827    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"}
Sep 08 13:32:22.208184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.207850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"}
Sep 08 13:32:22.447161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.447139    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:22.638837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638800    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638847    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638900    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638946    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.638979    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9bxp\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639003    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639033    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639090    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639141    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639175    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639205    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639237    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy\") pod \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\" (UID: \"8e5e1176-e7d9-48ba-864b-e495ff1c006d\") "
Sep 08 13:32:22.639325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639290    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:32:22.639890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639498    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-main-db\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.639985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.639958    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:22.641129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.641081    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:32:22.641870 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.641835    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.642352 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.642214    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp" (OuterVolumeSpecName: "kube-api-access-b9bxp") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "kube-api-access-b9bxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:32:22.642996 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.642964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume" (OuterVolumeSpecName: "config-volume") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.643092 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.643058    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.643431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.643405    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:32:22.643542 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.643523    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.643889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.643866    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out" (OuterVolumeSpecName: "config-out") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:32:22.644076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.644048    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.647163 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.647139    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.653795 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.653774    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config" (OuterVolumeSpecName: "web-config") pod "8e5e1176-e7d9-48ba-864b-e495ff1c006d" (UID: "8e5e1176-e7d9-48ba-864b-e495ff1c006d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:32:22.740261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740223    2583 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740257    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740268    2583 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-cluster-tls-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740276    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-metrics-client-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740289    2583 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-web-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740297    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9bxp\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-kube-api-access-b9bxp\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740307    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740315    2583 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/8e5e1176-e7d9-48ba-864b-e495ff1c006d-config-out\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740323    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-main-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740331    2583 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/8e5e1176-e7d9-48ba-864b-e495ff1c006d-tls-assets\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740338    2583 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e5e1176-e7d9-48ba-864b-e495ff1c006d-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:22.740479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:22.740346    2583 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/8e5e1176-e7d9-48ba-864b-e495ff1c006d-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:32:23.215668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215634    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323" exitCode=0
Sep 08 13:32:23.215668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215661    2583 generic.go:356] "Generic (PLEG): container finished" podID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerID="143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2" exitCode=0
Sep 08 13:32:23.216087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215712    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.216087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215717    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"}
Sep 08 13:32:23.216087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"}
Sep 08 13:32:23.216087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215821    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"8e5e1176-e7d9-48ba-864b-e495ff1c006d","Type":"ContainerDied","Data":"ed608ca54be25df136cb02c2885daaccae54f6287d0e205261da5e0b56510fa9"}
Sep 08 13:32:23.216087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.215858    2583 scope.go:117] "RemoveContainer" containerID="694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"
Sep 08 13:32:23.225120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.225100    2583 scope.go:117] "RemoveContainer" containerID="f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"
Sep 08 13:32:23.232950 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.232930    2583 scope.go:117] "RemoveContainer" containerID="83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"
Sep 08 13:32:23.239902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.239874    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:23.241058 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.241041    2583 scope.go:117] "RemoveContainer" containerID="143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"
Sep 08 13:32:23.244433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.244414    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:23.248878 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.248841    2583 scope.go:117] "RemoveContainer" containerID="0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"
Sep 08 13:32:23.256199 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.256181    2583 scope.go:117] "RemoveContainer" containerID="eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"
Sep 08 13:32:23.263669 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.263649    2583 scope.go:117] "RemoveContainer" containerID="a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"
Sep 08 13:32:23.270788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.270765    2583 scope.go:117] "RemoveContainer" containerID="694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"
Sep 08 13:32:23.271272 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.271192    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa\": container with ID starting with 694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa not found: ID does not exist" containerID="694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"
Sep 08 13:32:23.271378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.271274    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"} err="failed to get container status \"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa\": rpc error: code = NotFound desc = could not find container \"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa\": container with ID starting with 694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa not found: ID does not exist"
Sep 08 13:32:23.271378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.271298    2583 scope.go:117] "RemoveContainer" containerID="f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"
Sep 08 13:32:23.271692 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.271670    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323\": container with ID starting with f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323 not found: ID does not exist" containerID="f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"
Sep 08 13:32:23.271776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.271709    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"} err="failed to get container status \"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323\": rpc error: code = NotFound desc = could not find container \"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323\": container with ID starting with f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323 not found: ID does not exist"
Sep 08 13:32:23.271776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.271732    2583 scope.go:117] "RemoveContainer" containerID="83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"
Sep 08 13:32:23.272054 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.272033    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5\": container with ID starting with 83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5 not found: ID does not exist" containerID="83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"
Sep 08 13:32:23.272096 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272060    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"} err="failed to get container status \"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5\": rpc error: code = NotFound desc = could not find container \"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5\": container with ID starting with 83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5 not found: ID does not exist"
Sep 08 13:32:23.272096 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272076    2583 scope.go:117] "RemoveContainer" containerID="143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"
Sep 08 13:32:23.272353 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.272338    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2\": container with ID starting with 143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2 not found: ID does not exist" containerID="143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"
Sep 08 13:32:23.272397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272357    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"} err="failed to get container status \"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2\": rpc error: code = NotFound desc = could not find container \"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2\": container with ID starting with 143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2 not found: ID does not exist"
Sep 08 13:32:23.272397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272373    2583 scope.go:117] "RemoveContainer" containerID="0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"
Sep 08 13:32:23.272663 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.272645    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd\": container with ID starting with 0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd not found: ID does not exist" containerID="0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"
Sep 08 13:32:23.272727 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272669    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"} err="failed to get container status \"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd\": rpc error: code = NotFound desc = could not find container \"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd\": container with ID starting with 0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd not found: ID does not exist"
Sep 08 13:32:23.272727 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272683    2583 scope.go:117] "RemoveContainer" containerID="eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"
Sep 08 13:32:23.272826 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272789    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:23.272932 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.272911    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20\": container with ID starting with eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20 not found: ID does not exist" containerID="eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"
Sep 08 13:32:23.272972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272937    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"} err="failed to get container status \"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20\": rpc error: code = NotFound desc = could not find container \"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20\": container with ID starting with eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20 not found: ID does not exist"
Sep 08 13:32:23.272972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.272953    2583 scope.go:117] "RemoveContainer" containerID="a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"
Sep 08 13:32:23.273241 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:32:23.273206    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71\": container with ID starting with a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71 not found: ID does not exist" containerID="a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"
Sep 08 13:32:23.273241 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273235    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"} err="failed to get container status \"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71\": rpc error: code = NotFound desc = could not find container \"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71\": container with ID starting with a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71 not found: ID does not exist"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273249    2583 scope.go:117] "RemoveContainer" containerID="694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273252    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-metric"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273266    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-metric"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273279    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" containerName="console"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273284    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" containerName="console"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273313    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273321    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273331    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="init-config-reloader"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273339    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="init-config-reloader"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273348    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="config-reloader"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273353    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="config-reloader"
Sep 08 13:32:23.273357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273360    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="alertmanager"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273365    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="alertmanager"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273374    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-web"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273382    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-web"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273391    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="prom-label-proxy"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273399    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="prom-label-proxy"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273467    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-web"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273482    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="prom-label-proxy"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273491    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy-metric"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273503    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="556f2462-f3bd-47cc-a247-abb6fdcf14ed" containerName="console"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273512    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="config-reloader"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273509    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa"} err="failed to get container status \"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa\": rpc error: code = NotFound desc = could not find container \"694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa\": container with ID starting with 694f2393b9f167d91b8ca8a796a071f7e5992d12e881e4d54f3324cd6d3905fa not found: ID does not exist"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273531    2583 scope.go:117] "RemoveContainer" containerID="f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273519    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="kube-rbac-proxy"
Sep 08 13:32:23.273816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273608    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" containerName="alertmanager"
Sep 08 13:32:23.274279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273887    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323"} err="failed to get container status \"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323\": rpc error: code = NotFound desc = could not find container \"f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323\": container with ID starting with f46f91a35b23a8c387b127353cd0eed4f7ae6bceef71e953818ad71c1775a323 not found: ID does not exist"
Sep 08 13:32:23.274279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.273911    2583 scope.go:117] "RemoveContainer" containerID="83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"
Sep 08 13:32:23.274279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274164    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5"} err="failed to get container status \"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5\": rpc error: code = NotFound desc = could not find container \"83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5\": container with ID starting with 83845be5be442c178e8194305de8fa26737a673771c14d9354abe7d01e13cde5 not found: ID does not exist"
Sep 08 13:32:23.274279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274181    2583 scope.go:117] "RemoveContainer" containerID="143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"
Sep 08 13:32:23.274455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274440    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2"} err="failed to get container status \"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2\": rpc error: code = NotFound desc = could not find container \"143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2\": container with ID starting with 143775859d1014fc19175bcb1666197bf883292b7f5bf6f985134f2e9c372ce2 not found: ID does not exist"
Sep 08 13:32:23.274455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274455    2583 scope.go:117] "RemoveContainer" containerID="0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"
Sep 08 13:32:23.274683 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274667    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd"} err="failed to get container status \"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd\": rpc error: code = NotFound desc = could not find container \"0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd\": container with ID starting with 0646d1ded03a2b735eb11fd2fce56769ea0d98d6771c8b81d2c7c6817524d6dd not found: ID does not exist"
Sep 08 13:32:23.274776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274683    2583 scope.go:117] "RemoveContainer" containerID="eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"
Sep 08 13:32:23.274907 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274893    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20"} err="failed to get container status \"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20\": rpc error: code = NotFound desc = could not find container \"eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20\": container with ID starting with eba072dc3d654d14cbe9c58c1f729054e81a5e9b3bd34fd746deed03b91bfc20 not found: ID does not exist"
Sep 08 13:32:23.274944 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.274907    2583 scope.go:117] "RemoveContainer" containerID="a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"
Sep 08 13:32:23.275124 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.275107    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71"} err="failed to get container status \"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71\": rpc error: code = NotFound desc = could not find container \"a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71\": container with ID starting with a10441f6072d99ec4f6bd17e9427d459906c7d1b897ba9f780139ad98580ad71 not found: ID does not exist"
Sep 08 13:32:23.277281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.277268    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.279315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279297    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 08 13:32:23.279391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279366    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 08 13:32:23.279391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279385    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 08 13:32:23.279497 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279465    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 08 13:32:23.279497 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279468    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 08 13:32:23.279642 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279612    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 08 13:32:23.279855 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279840    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 08 13:32:23.279938 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.279926    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 08 13:32:23.280346 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.280334    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-q57nm\""
Sep 08 13:32:23.285578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.285379    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 08 13:32:23.291593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.291566    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:23.448435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448400    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448438    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448460    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448540    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448631    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-out\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448695    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448712    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448739    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-web-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448761    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.448830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.448778    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wpcd\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-kube-api-access-4wpcd\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549155 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549123    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549161    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wpcd\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-kube-api-access-4wpcd\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549187    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549216    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549245    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549282    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549308    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549541 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549580    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549612    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-out\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549637    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549797 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549666    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.549797 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.549698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-web-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.550657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.550181    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.550657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.550528    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.551366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.551339    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.552593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.552528    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-out\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.552698 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.552614    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.552861 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.552828    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.552933 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.552872    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-web-config\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.553259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.553235    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.553370 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.553271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.553733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.553708    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.553870 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.553855    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.554146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.554132    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-config-volume\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.557038 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.557018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wpcd\" (UniqueName: \"kubernetes.io/projected/e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f-kube-api-access-4wpcd\") pod \"alertmanager-main-0\" (UID: \"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.588127 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.588091    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 08 13:32:23.865333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:23.865306    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 08 13:32:23.866940 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:32:23.866914    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7b8a25b_9ac9_4f8b_8cd0_c5c5b963d86f.slice/crio-9c5c257607adea9b9da6d76da4a68b5a9bcdf9b8bcf2d5475750d5e6c48c2074 WatchSource:0}: Error finding container 9c5c257607adea9b9da6d76da4a68b5a9bcdf9b8bcf2d5475750d5e6c48c2074: Status 404 returned error can't find the container with id 9c5c257607adea9b9da6d76da4a68b5a9bcdf9b8bcf2d5475750d5e6c48c2074
Sep 08 13:32:24.222063 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:24.221981    2583 generic.go:356] "Generic (PLEG): container finished" podID="e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f" containerID="eb769c47c7d9276fc7e4f3d6b2a6b165870585fcff5eccd25e9dc0892df63d2e" exitCode=0
Sep 08 13:32:24.222063 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:24.222028    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerDied","Data":"eb769c47c7d9276fc7e4f3d6b2a6b165870585fcff5eccd25e9dc0892df63d2e"}
Sep 08 13:32:24.222433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:24.222075    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"9c5c257607adea9b9da6d76da4a68b5a9bcdf9b8bcf2d5475750d5e6c48c2074"}
Sep 08 13:32:24.230276 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:24.230245    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8e5e1176-e7d9-48ba-864b-e495ff1c006d" path="/var/lib/kubelet/pods/8e5e1176-e7d9-48ba-864b-e495ff1c006d/volumes"
Sep 08 13:32:25.161298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.161259    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-758c7d646f-2rl47"]
Sep 08 13:32:25.164319 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.164303    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.166791 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.166769    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 08 13:32:25.166889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.166817    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 08 13:32:25.166889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.166883    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 08 13:32:25.167004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.166989    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 08 13:32:25.167052 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.167006    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-mrrfm\""
Sep 08 13:32:25.167142 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.167125    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 08 13:32:25.172744 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.172724    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 08 13:32:25.180880 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.180858    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-758c7d646f-2rl47"]
Sep 08 13:32:25.230257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230223    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"8f073e0bbd5c9de226155fd8f28f2f481a524166ca3d0af3c767dd0ce8fa0ed8"}
Sep 08 13:32:25.230696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230288    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"c8fd11385140a8db198a1912262258fe34bed07fc10c702e757a50c6b4b3802b"}
Sep 08 13:32:25.230696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230320    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"a92f610086b19b15293cbdaabdf6538a23e503ef14842121e3e979c603fc33cc"}
Sep 08 13:32:25.230696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230345    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"9ea911f23330658782f68f1ee70e86d2d60b8c957a738523791c291ac18781c3"}
Sep 08 13:32:25.230696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"49a4046481baf7664d0a65b240da60774021a0b68ce485447528dae40ae7890f"}
Sep 08 13:32:25.230696 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.230389    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f","Type":"ContainerStarted","Data":"16b0962c5068a5f0362b7acea09b6e374924f92814d115e4a1066d9b32ea80ea"}
Sep 08 13:32:25.254234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.254183    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.238073606 podStartE2EDuration="2.254167181s" totalImagesPullingTime="0s" totalInitContainerRuntime="16.093575ms" isStatefulPod=false podCreationTimestamp="2026-09-08 13:32:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:32:25.252072014 +0000 UTC m=+231.469988338" watchObservedRunningTime="2026-09-08 13:32:25.254167181 +0000 UTC m=+231.472083505"
Sep 08 13:32:25.265209 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265167    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265266    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265297    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-federate-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265322    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-serving-certs-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265591 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265422    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nch66\" (UniqueName: \"kubernetes.io/projected/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-kube-api-access-nch66\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265591 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.265706 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265636    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.266019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.265972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-metrics-client-ca\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.366942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.366893    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.366942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.366947    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.366982    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-metrics-client-ca\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367011    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367042    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-federate-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367344 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367291    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-serving-certs-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367376    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nch66\" (UniqueName: \"kubernetes.io/projected/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-kube-api-access-nch66\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.367986 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.367898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-metrics-client-ca\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.368271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.368141    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.368271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.368148    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-serving-certs-ca-bundle\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.369835 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.369804    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.369941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.369888    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-federate-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.370007 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.369979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-telemeter-client-tls\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.370007 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.369987    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-secret-telemeter-client\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.374781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.374761    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nch66\" (UniqueName: \"kubernetes.io/projected/acb517ae-2729-4d6a-b4ca-e0a63ffc77fa-kube-api-access-nch66\") pod \"telemeter-client-758c7d646f-2rl47\" (UID: \"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa\") " pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.474963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.474868    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47"
Sep 08 13:32:25.756207 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:25.756137    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-758c7d646f-2rl47"]
Sep 08 13:32:25.759357 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:32:25.759328    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podacb517ae_2729_4d6a_b4ca_e0a63ffc77fa.slice/crio-82858caa68b00bb4e023e8c8b41257147d09eed3d3cd8ebbe59abef92e423178 WatchSource:0}: Error finding container 82858caa68b00bb4e023e8c8b41257147d09eed3d3cd8ebbe59abef92e423178: Status 404 returned error can't find the container with id 82858caa68b00bb4e023e8c8b41257147d09eed3d3cd8ebbe59abef92e423178
Sep 08 13:32:26.234618 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:26.234584    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47" event={"ID":"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa","Type":"ContainerStarted","Data":"82858caa68b00bb4e023e8c8b41257147d09eed3d3cd8ebbe59abef92e423178"}
Sep 08 13:32:26.273179 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:26.273141    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:26.332726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:26.332693    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:28.246518 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.246486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47" event={"ID":"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa","Type":"ContainerStarted","Data":"8818c084617fe6edf2c705b2796e295603ffb881ef0d743cdaff14f08548db5b"}
Sep 08 13:32:28.246934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.246538    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47" event={"ID":"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa","Type":"ContainerStarted","Data":"8590150d4d001e285cfc003b47350f94ef5afdd892e7f4be0633666bf5227f08"}
Sep 08 13:32:28.246934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.246660    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47" event={"ID":"acb517ae-2729-4d6a-b4ca-e0a63ffc77fa","Type":"ContainerStarted","Data":"b43ffc8b65ecef8e41bb953bc778ee5223dce22c861595c802d3b6eac3f6739c"}
Sep 08 13:32:28.266711 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.266662    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-758c7d646f-2rl47" podStartSLOduration=1.564901501 podStartE2EDuration="3.266648059s" totalImagesPullingTime="1.701746558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:32:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:32:28.265375461 +0000 UTC m=+234.483291799" watchObservedRunningTime="2026-09-08 13:32:28.266648059 +0000 UTC m=+234.484564383"
Sep 08 13:32:28.352216 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.352181    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:32:28.354509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.354487    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.365825 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.365795    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:32:28.391622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391832 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391640    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqrk4\" (UniqueName: \"kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391832 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391683    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391832 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391753    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391832 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.391832 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.391799    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.403746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.403711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:28.492764 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492730    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492764 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492765    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492785    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqrk4\" (UniqueName: \"kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492806    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492866    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492894    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.492997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.492927    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.493720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.493642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.493720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.493681    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.493720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.493682    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.493962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.493759    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.495474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.495442    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.495598 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.495505    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.501125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.501061    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqrk4\" (UniqueName: \"kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4\") pod \"console-7954b4658f-n7pfw\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") " pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.663705 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.663665    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:28.938078 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:28.938042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:32:28.941124 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:32:28.941095    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b43ae90_2e45_4969_a5e2_55b4b70bd782.slice/crio-f3515c50adf5fd5a2a7d391b0992dbdd5389990a41e4869ce95acac375ef42a6 WatchSource:0}: Error finding container f3515c50adf5fd5a2a7d391b0992dbdd5389990a41e4869ce95acac375ef42a6: Status 404 returned error can't find the container with id f3515c50adf5fd5a2a7d391b0992dbdd5389990a41e4869ce95acac375ef42a6
Sep 08 13:32:29.252415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:29.252315    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7954b4658f-n7pfw" event={"ID":"0b43ae90-2e45-4969-a5e2-55b4b70bd782","Type":"ContainerStarted","Data":"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"}
Sep 08 13:32:29.252415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:29.252366    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7954b4658f-n7pfw" event={"ID":"0b43ae90-2e45-4969-a5e2-55b4b70bd782","Type":"ContainerStarted","Data":"f3515c50adf5fd5a2a7d391b0992dbdd5389990a41e4869ce95acac375ef42a6"}
Sep 08 13:32:29.271302 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:29.271250    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7954b4658f-n7pfw" podStartSLOduration=1.271233233 podStartE2EDuration="1.271233233s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:32:29.270276693 +0000 UTC m=+235.488193045" watchObservedRunningTime="2026-09-08 13:32:29.271233233 +0000 UTC m=+235.489149558"
Sep 08 13:32:29.790232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:29.790195    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:38.664862 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:38.664829    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:38.665234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:38.664888    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:38.669466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:38.669448    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:39.298459 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:39.298430    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:32:39.339387 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:39.339348    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:32:40.453958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:40.453904    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:32:40.510741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:32:40.510706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:04.365093 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.365001    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-68f7fbbb55-swbzd" podUID="fc32a5f8-f765-465f-a542-fb3803cb603f" containerName="console" containerID="cri-o://17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710" gracePeriod=15
Sep 08 13:33:04.596740 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.596716    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:33:04.709529 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709428    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5p9w\" (UniqueName: \"kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709529 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709519    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709529 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709537    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709657    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709707    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709739    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.709837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709778    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config\") pod \"fc32a5f8-f765-465f-a542-fb3803cb603f\" (UID: \"fc32a5f8-f765-465f-a542-fb3803cb603f\") "
Sep 08 13:33:04.710046 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.709975    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:33:04.710139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.710115    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:33:04.710233 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.710194    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:33:04.710380 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.710320    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config" (OuterVolumeSpecName: "console-config") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:33:04.711948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.711927    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:33:04.712059 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.711957    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w" (OuterVolumeSpecName: "kube-api-access-g5p9w") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "kube-api-access-g5p9w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:33:04.712121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.712081    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc32a5f8-f765-465f-a542-fb3803cb603f" (UID: "fc32a5f8-f765-465f-a542-fb3803cb603f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:33:04.810424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810391    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810416    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810426    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5p9w\" (UniqueName: \"kubernetes.io/projected/fc32a5f8-f765-465f-a542-fb3803cb603f-kube-api-access-g5p9w\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810434    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810442    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810450    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc32a5f8-f765-465f-a542-fb3803cb603f-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:04.810685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:04.810457    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc32a5f8-f765-465f-a542-fb3803cb603f-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:33:05.395716 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.395687    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc32a5f8-f765-465f-a542-fb3803cb603f" containerID="17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710" exitCode=0
Sep 08 13:33:05.396161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.395733    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68f7fbbb55-swbzd"
Sep 08 13:33:05.396161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.395786    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f7fbbb55-swbzd" event={"ID":"fc32a5f8-f765-465f-a542-fb3803cb603f","Type":"ContainerDied","Data":"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"}
Sep 08 13:33:05.396161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.395839    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68f7fbbb55-swbzd" event={"ID":"fc32a5f8-f765-465f-a542-fb3803cb603f","Type":"ContainerDied","Data":"7489619cf5135f7f5173fb60bf71baebf10cf377312ec6deb090d3d101aba2ee"}
Sep 08 13:33:05.396161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.395871    2583 scope.go:117] "RemoveContainer" containerID="17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"
Sep 08 13:33:05.404689 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.404673    2583 scope.go:117] "RemoveContainer" containerID="17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"
Sep 08 13:33:05.404944 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:33:05.404930    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710\": container with ID starting with 17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710 not found: ID does not exist" containerID="17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"
Sep 08 13:33:05.404988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.404950    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710"} err="failed to get container status \"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710\": rpc error: code = NotFound desc = could not find container \"17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710\": container with ID starting with 17ced6682bc1a7b9fe58892a7a04cded9f88850c9128fb244eb604a1deb67710 not found: ID does not exist"
Sep 08 13:33:05.415470 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.415445    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:33:05.416978 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:05.416959    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68f7fbbb55-swbzd"]
Sep 08 13:33:06.230148 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:06.230115    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc32a5f8-f765-465f-a542-fb3803cb603f" path="/var/lib/kubelet/pods/fc32a5f8-f765-465f-a542-fb3803cb603f/volumes"
Sep 08 13:33:15.009073 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:15.009028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:34.144132 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:34.144105    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:33:34.145867 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:34.145847    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:33:34.148143 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:34.148125    2583 kubelet.go:1648] "Image garbage collection succeeded"
Sep 08 13:33:35.299229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:35.299189    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:40.451381 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.451348    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:33:40.451881 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.451821    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc32a5f8-f765-465f-a542-fb3803cb603f" containerName="console"
Sep 08 13:33:40.451881 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.451838    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc32a5f8-f765-465f-a542-fb3803cb603f" containerName="console"
Sep 08 13:33:40.452002 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.451928    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc32a5f8-f765-465f-a542-fb3803cb603f" containerName="console"
Sep 08 13:33:40.455799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.455779    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.477057 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.477021    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:33:40.497617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497614    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497886 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497828    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497886 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497878    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-82v6s\" (UniqueName: \"kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.497953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.497899    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599072    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-82v6s\" (UniqueName: \"kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599121    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599156    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599171    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599252    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599301    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.599981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.599955    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.600085 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.600042    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.600279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.600259    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.600445 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.600425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.601806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.601784    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.601806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.601794    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.606914 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.606895    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-82v6s\" (UniqueName: \"kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s\") pod \"console-59d7bf9f86-5jxk4\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") " pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:40.767218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:40.767124    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:41.040034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.039947    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:33:41.042719 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:33:41.042690    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff3f88ae_4cc5_4424_ba4a_11c2c37d1c57.slice/crio-09a4dab78268c7747f1c84360520ed161abfadcdb614169f65aae68b230a62b5 WatchSource:0}: Error finding container 09a4dab78268c7747f1c84360520ed161abfadcdb614169f65aae68b230a62b5: Status 404 returned error can't find the container with id 09a4dab78268c7747f1c84360520ed161abfadcdb614169f65aae68b230a62b5
Sep 08 13:33:41.452209 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.452126    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:41.503778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.503750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:41.540995 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.540964    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7bf9f86-5jxk4" event={"ID":"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57","Type":"ContainerStarted","Data":"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"}
Sep 08 13:33:41.541159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.541009    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7bf9f86-5jxk4" event={"ID":"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57","Type":"ContainerStarted","Data":"09a4dab78268c7747f1c84360520ed161abfadcdb614169f65aae68b230a62b5"}
Sep 08 13:33:41.558851 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:41.558804    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-59d7bf9f86-5jxk4" podStartSLOduration=1.558790513 podStartE2EDuration="1.558790513s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:33:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:33:41.557336952 +0000 UTC m=+307.775253288" watchObservedRunningTime="2026-09-08 13:33:41.558790513 +0000 UTC m=+307.776706837"
Sep 08 13:33:47.171558 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:47.171519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:47.228975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:47.228940    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:50.767751 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:50.767360    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:50.768314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:50.768295    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:50.776681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:50.776653    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:51.585892 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:51.585864    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:33:51.632176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:51.632146    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:33:51.728335 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:51.728304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:51.783260 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:51.783220    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:51.843729 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:51.843635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:57.666841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:57.666805    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:33:57.767648 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:33:57.767611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:34:07.437874 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:07.437835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:34:16.652695 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.652652    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7954b4658f-n7pfw" podUID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" containerName="console" containerID="cri-o://72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78" gracePeriod=15
Sep 08 13:34:16.897069 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897039    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:34:16.897369 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897351    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897454 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897385    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqrk4\" (UniqueName: \"kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897511 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897511 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897483    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897633 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897523    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897633 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897578    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897731 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897702    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:34:16.897785 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897754    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config\") pod \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\" (UID: \"0b43ae90-2e45-4969-a5e2-55b4b70bd782\") "
Sep 08 13:34:16.897991 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.897964    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:34:16.898176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.898038    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:34:16.898176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.898139    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.898176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.898160    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.898176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.898173    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.898478 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.898309    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config" (OuterVolumeSpecName: "console-config") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:34:16.899769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.899732    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:34:16.899868 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.899798    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4" (OuterVolumeSpecName: "kube-api-access-lqrk4") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "kube-api-access-lqrk4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:34:16.899868 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.899829    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0b43ae90-2e45-4969-a5e2-55b4b70bd782" (UID: "0b43ae90-2e45-4969-a5e2-55b4b70bd782"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:34:16.999450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.999412    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.999450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.999439    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.999450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.999448    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b43ae90-2e45-4969-a5e2-55b4b70bd782-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:16.999450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:16.999457    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqrk4\" (UniqueName: \"kubernetes.io/projected/0b43ae90-2e45-4969-a5e2-55b4b70bd782-kube-api-access-lqrk4\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:34:17.687039 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.687003    2583 generic.go:356] "Generic (PLEG): container finished" podID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" containerID="72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78" exitCode=0
Sep 08 13:34:17.687425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.687049    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7954b4658f-n7pfw"
Sep 08 13:34:17.687425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.687091    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7954b4658f-n7pfw" event={"ID":"0b43ae90-2e45-4969-a5e2-55b4b70bd782","Type":"ContainerDied","Data":"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"}
Sep 08 13:34:17.687425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.687148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7954b4658f-n7pfw" event={"ID":"0b43ae90-2e45-4969-a5e2-55b4b70bd782","Type":"ContainerDied","Data":"f3515c50adf5fd5a2a7d391b0992dbdd5389990a41e4869ce95acac375ef42a6"}
Sep 08 13:34:17.687425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.687170    2583 scope.go:117] "RemoveContainer" containerID="72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"
Sep 08 13:34:17.696422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.696401    2583 scope.go:117] "RemoveContainer" containerID="72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"
Sep 08 13:34:17.696735 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:34:17.696714    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78\": container with ID starting with 72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78 not found: ID does not exist" containerID="72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"
Sep 08 13:34:17.696825 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.696748    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78"} err="failed to get container status \"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78\": rpc error: code = NotFound desc = could not find container \"72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78\": container with ID starting with 72b0c59da50b521d153a2d444c0db2ae2c7f4faffb4682569d2a1f2f2fd85e78 not found: ID does not exist"
Sep 08 13:34:17.707337 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.707309    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:34:17.711448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:17.711424    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7954b4658f-n7pfw"]
Sep 08 13:34:18.229816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:34:18.229777    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" path="/var/lib/kubelet/pods/0b43ae90-2e45-4969-a5e2-55b4b70bd782/volumes"
Sep 08 13:35:03.748593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:03.748541    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:03.841427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:03.841395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:04.401149 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.401120    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"]
Sep 08 13:35:04.401458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.401446    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" containerName="console"
Sep 08 13:35:04.401507 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.401460    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" containerName="console"
Sep 08 13:35:04.401545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.401512    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b43ae90-2e45-4969-a5e2-55b4b70bd782" containerName="console"
Sep 08 13:35:04.404659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.404634    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.407177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.407155    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 13:35:04.407705 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.407682    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pdttm\""
Sep 08 13:35:04.407822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.407707    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 13:35:04.412304 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.412277    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"]
Sep 08 13:35:04.503988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.503956    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkdjf\" (UniqueName: \"kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.503988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.503991    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.504188 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.504104    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.605284 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.605232    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.605448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.605302    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xkdjf\" (UniqueName: \"kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.605448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.605329    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.605644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.605625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.605738 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.605717    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.613801 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.613780    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkdjf\" (UniqueName: \"kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.714422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.714344    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:04.982498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.982470    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"]
Sep 08 13:35:04.984735 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:35:04.984704    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod169f803e_e1a6_43b7_96bf_415c18492d57.slice/crio-9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9 WatchSource:0}: Error finding container 9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9: Status 404 returned error can't find the container with id 9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9
Sep 08 13:35:04.986801 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:04.986774    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:35:05.883779 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:05.883741    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerStarted","Data":"9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9"}
Sep 08 13:35:11.910929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:11.910895    2583 generic.go:356] "Generic (PLEG): container finished" podID="169f803e-e1a6-43b7-96bf-415c18492d57" containerID="10decfc1415e08823b81c5b7745bf45bb129a2e069c056c70b8f84a20259004a" exitCode=0
Sep 08 13:35:11.911312 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:11.910986    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerDied","Data":"10decfc1415e08823b81c5b7745bf45bb129a2e069c056c70b8f84a20259004a"}
Sep 08 13:35:13.922643 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:13.922606    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerStarted","Data":"dfb7e7bbe6ecfdb0ebfec2380361e2060d07c9c997a18b25f9e40178a4ded755"}
Sep 08 13:35:14.928171 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:14.928134    2583 generic.go:356] "Generic (PLEG): container finished" podID="169f803e-e1a6-43b7-96bf-415c18492d57" containerID="dfb7e7bbe6ecfdb0ebfec2380361e2060d07c9c997a18b25f9e40178a4ded755" exitCode=0
Sep 08 13:35:14.928567 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:14.928190    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerDied","Data":"dfb7e7bbe6ecfdb0ebfec2380361e2060d07c9c997a18b25f9e40178a4ded755"}
Sep 08 13:35:22.966110 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:22.966074    2583 generic.go:356] "Generic (PLEG): container finished" podID="169f803e-e1a6-43b7-96bf-415c18492d57" containerID="3bead411dfe69b86a3095932d4296afaedc3d99396eb6a241c9b4191ab7450fc" exitCode=0
Sep 08 13:35:22.966464 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:22.966139    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerDied","Data":"3bead411dfe69b86a3095932d4296afaedc3d99396eb6a241c9b4191ab7450fc"}
Sep 08 13:35:24.130964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.130924    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:24.180402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.180364    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkdjf\" (UniqueName: \"kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf\") pod \"169f803e-e1a6-43b7-96bf-415c18492d57\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") "
Sep 08 13:35:24.180644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.180419    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle\") pod \"169f803e-e1a6-43b7-96bf-415c18492d57\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") "
Sep 08 13:35:24.180644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.180454    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util\") pod \"169f803e-e1a6-43b7-96bf-415c18492d57\" (UID: \"169f803e-e1a6-43b7-96bf-415c18492d57\") "
Sep 08 13:35:24.181143 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.181119    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle" (OuterVolumeSpecName: "bundle") pod "169f803e-e1a6-43b7-96bf-415c18492d57" (UID: "169f803e-e1a6-43b7-96bf-415c18492d57"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:35:24.182863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.182840    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf" (OuterVolumeSpecName: "kube-api-access-xkdjf") pod "169f803e-e1a6-43b7-96bf-415c18492d57" (UID: "169f803e-e1a6-43b7-96bf-415c18492d57"). InnerVolumeSpecName "kube-api-access-xkdjf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:35:24.184494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.184469    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util" (OuterVolumeSpecName: "util") pod "169f803e-e1a6-43b7-96bf-415c18492d57" (UID: "169f803e-e1a6-43b7-96bf-415c18492d57"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:35:24.281136 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.281104    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:24.281136 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.281136    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/169f803e-e1a6-43b7-96bf-415c18492d57-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:24.281375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.281152    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkdjf\" (UniqueName: \"kubernetes.io/projected/169f803e-e1a6-43b7-96bf-415c18492d57-kube-api-access-xkdjf\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:24.975661 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.975624    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz"
Sep 08 13:35:24.975661 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.975646    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n24bz" event={"ID":"169f803e-e1a6-43b7-96bf-415c18492d57","Type":"ContainerDied","Data":"9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9"}
Sep 08 13:35:24.975931 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:24.975693    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9e229bb7c99591f3d412cf6a0c9f87070faa5697dd8dfe374a0eec95e5ce4ae9"
Sep 08 13:35:30.444691 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:30.444569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:31.249059 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:31.249013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:32.401905 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.401873    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"]
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402181    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="pull"
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402196    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="pull"
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402219    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="util"
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402229    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="util"
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402243    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="extract"
Sep 08 13:35:32.402296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402253    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="extract"
Sep 08 13:35:32.402604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.402345    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="169f803e-e1a6-43b7-96bf-415c18492d57" containerName="extract"
Sep 08 13:35:32.404745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.404728    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.407203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.407183    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 08 13:35:32.407331 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.407181    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-x6ms9\""
Sep 08 13:35:32.407331 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.407298    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:35:32.420777 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.420757    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"]
Sep 08 13:35:32.554999 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.554960    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k79sx\" (UniqueName: \"kubernetes.io/projected/885273e3-4091-4735-883c-6a184a016cea-kube-api-access-k79sx\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.555169 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.555032    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885273e3-4091-4735-883c-6a184a016cea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.655987 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.655918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885273e3-4091-4735-883c-6a184a016cea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.656094 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.655988    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k79sx\" (UniqueName: \"kubernetes.io/projected/885273e3-4091-4735-883c-6a184a016cea-kube-api-access-k79sx\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.656314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.656296    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/885273e3-4091-4735-883c-6a184a016cea-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.668335 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.668314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k79sx\" (UniqueName: \"kubernetes.io/projected/885273e3-4091-4735-883c-6a184a016cea-kube-api-access-k79sx\") pod \"cert-manager-operator-controller-manager-55c444bf86-tpsxl\" (UID: \"885273e3-4091-4735-883c-6a184a016cea\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.713921 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.713898    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"
Sep 08 13:35:32.989967 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:32.989942    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl"]
Sep 08 13:35:32.992836 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:35:32.992813    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod885273e3_4091_4735_883c_6a184a016cea.slice/crio-c2b19a23db582ca3470887d38ab1795d54aa4bbaac5c2e3d3096fdedc3ede8b9 WatchSource:0}: Error finding container c2b19a23db582ca3470887d38ab1795d54aa4bbaac5c2e3d3096fdedc3ede8b9: Status 404 returned error can't find the container with id c2b19a23db582ca3470887d38ab1795d54aa4bbaac5c2e3d3096fdedc3ede8b9
Sep 08 13:35:33.010095 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:33.010068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl" event={"ID":"885273e3-4091-4735-883c-6a184a016cea","Type":"ContainerStarted","Data":"c2b19a23db582ca3470887d38ab1795d54aa4bbaac5c2e3d3096fdedc3ede8b9"}
Sep 08 13:35:34.061676 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:34.061638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:34.161871 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:34.161843    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:38.646411 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:38.646364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:38.703561 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:38.703525    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:45.067401 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.067361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl" event={"ID":"885273e3-4091-4735-883c-6a184a016cea","Type":"ContainerStarted","Data":"675acec082c856b0586ec7fe3ea6102ba9ac4a2d92290f4a4b3c261cdf6e0b0c"}
Sep 08 13:35:45.088177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.088128    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-tpsxl" podStartSLOduration=1.686190107 podStartE2EDuration="13.088110915s" totalImagesPullingTime="11.401920808s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:35:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:35:45.086470826 +0000 UTC m=+431.304387150" watchObservedRunningTime="2026-09-08 13:35:45.088110915 +0000 UTC m=+431.306027308"
Sep 08 13:35:45.233335 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.233301    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:45.360595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.360483    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:45.484029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.483990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:45.582539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.582499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:45.635687 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.635610    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"]
Sep 08 13:35:45.637998 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.637979    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.641118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.641092    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pdttm\""
Sep 08 13:35:45.641261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.641139    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 13:35:45.642645 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.642626    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 13:35:45.651985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.651953    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"]
Sep 08 13:35:45.771324 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.771277    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.771509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.771364    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h96hz\" (UniqueName: \"kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.771509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.771402    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.871910 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.871872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.872080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.871957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.872080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.872012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h96hz\" (UniqueName: \"kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.872350 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.872329    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.872389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.872345    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.880590 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.880535    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h96hz\" (UniqueName: \"kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:45.947468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:45.947385    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:46.224805 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:46.224779    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"]
Sep 08 13:35:47.077956 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.077921    2583 generic.go:356] "Generic (PLEG): container finished" podID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerID="634625bf43e178393657e755724432c8cabfc2f7c6623a89018fb2d4a68ec291" exitCode=0
Sep 08 13:35:47.078150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.078027    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz" event={"ID":"0224c869-4b50-4d70-853d-c1f2684dbdbb","Type":"ContainerDied","Data":"634625bf43e178393657e755724432c8cabfc2f7c6623a89018fb2d4a68ec291"}
Sep 08 13:35:47.078150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.078078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz" event={"ID":"0224c869-4b50-4d70-853d-c1f2684dbdbb","Type":"ContainerStarted","Data":"586bf4707a29495834aae01d61b9cfe029123d88d1c4843e9b81c82a8115721b"}
Sep 08 13:35:47.436019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.435982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:47.622051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.622020    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:47.797483 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.797445    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"]
Sep 08 13:35:47.800385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.800357    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:47.802763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.802667    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 08 13:35:47.802763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.802710    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 08 13:35:47.802763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.802710    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-lmnrf\""
Sep 08 13:35:47.807106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.807081    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"]
Sep 08 13:35:47.990150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.990116    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7fct\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-kube-api-access-k7fct\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:47.990328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:47.990237    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.091479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.091400    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7fct\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-kube-api-access-k7fct\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.091653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.091531    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.100403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.100369    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.100724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.100701    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7fct\" (UniqueName: \"kubernetes.io/projected/ca373978-0202-4f23-9f50-62660c8152f2-kube-api-access-k7fct\") pod \"cert-manager-cainjector-559cdd8c87-wwrkd\" (UID: \"ca373978-0202-4f23-9f50-62660c8152f2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.127630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.127592    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"
Sep 08 13:35:48.907958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:48.907932    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd"]
Sep 08 13:35:48.910209 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:35:48.910184    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podca373978_0202_4f23_9f50_62660c8152f2.slice/crio-c2dd8eec360eff24ef78d13cff8185274a438eb61a95ab464f18503199c7e2ed WatchSource:0}: Error finding container c2dd8eec360eff24ef78d13cff8185274a438eb61a95ab464f18503199c7e2ed: Status 404 returned error can't find the container with id c2dd8eec360eff24ef78d13cff8185274a438eb61a95ab464f18503199c7e2ed
Sep 08 13:35:49.088395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:49.088359    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd" event={"ID":"ca373978-0202-4f23-9f50-62660c8152f2","Type":"ContainerStarted","Data":"c2dd8eec360eff24ef78d13cff8185274a438eb61a95ab464f18503199c7e2ed"}
Sep 08 13:35:49.090183 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:49.090156    2583 generic.go:356] "Generic (PLEG): container finished" podID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerID="63c511a1694faf19b6b5c7ae53cbe84774cb11d7cd1a5f01cfee6e4236ac336c" exitCode=0
Sep 08 13:35:49.090298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:49.090239    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz" event={"ID":"0224c869-4b50-4d70-853d-c1f2684dbdbb","Type":"ContainerDied","Data":"63c511a1694faf19b6b5c7ae53cbe84774cb11d7cd1a5f01cfee6e4236ac336c"}
Sep 08 13:35:50.097776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:50.097745    2583 generic.go:356] "Generic (PLEG): container finished" podID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerID="8430d1e0c71b6255e3046a745d8d6fc3659eb62a0aa4f206fbe4859471ab4717" exitCode=0
Sep 08 13:35:50.098156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:50.097834    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz" event={"ID":"0224c869-4b50-4d70-853d-c1f2684dbdbb","Type":"ContainerDied","Data":"8430d1e0c71b6255e3046a745d8d6fc3659eb62a0aa4f206fbe4859471ab4717"}
Sep 08 13:35:51.256401 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.256358    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:51.318789 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.318757    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util\") pod \"0224c869-4b50-4d70-853d-c1f2684dbdbb\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") "
Sep 08 13:35:51.318972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.318810    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h96hz\" (UniqueName: \"kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz\") pod \"0224c869-4b50-4d70-853d-c1f2684dbdbb\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") "
Sep 08 13:35:51.318972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.318882    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle\") pod \"0224c869-4b50-4d70-853d-c1f2684dbdbb\" (UID: \"0224c869-4b50-4d70-853d-c1f2684dbdbb\") "
Sep 08 13:35:51.319360 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.319319    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle" (OuterVolumeSpecName: "bundle") pod "0224c869-4b50-4d70-853d-c1f2684dbdbb" (UID: "0224c869-4b50-4d70-853d-c1f2684dbdbb"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:35:51.321660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.321630    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz" (OuterVolumeSpecName: "kube-api-access-h96hz") pod "0224c869-4b50-4d70-853d-c1f2684dbdbb" (UID: "0224c869-4b50-4d70-853d-c1f2684dbdbb"). InnerVolumeSpecName "kube-api-access-h96hz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:35:51.325106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.325063    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util" (OuterVolumeSpecName: "util") pod "0224c869-4b50-4d70-853d-c1f2684dbdbb" (UID: "0224c869-4b50-4d70-853d-c1f2684dbdbb"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:35:51.420139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.420055    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:51.420139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.420107    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h96hz\" (UniqueName: \"kubernetes.io/projected/0224c869-4b50-4d70-853d-c1f2684dbdbb-kube-api-access-h96hz\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:51.420139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:51.420122    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0224c869-4b50-4d70-853d-c1f2684dbdbb-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:35:52.111888 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:52.111861    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz"
Sep 08 13:35:52.112074 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:52.111860    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87ff7mcz" event={"ID":"0224c869-4b50-4d70-853d-c1f2684dbdbb","Type":"ContainerDied","Data":"586bf4707a29495834aae01d61b9cfe029123d88d1c4843e9b81c82a8115721b"}
Sep 08 13:35:52.112074 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:52.112015    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="586bf4707a29495834aae01d61b9cfe029123d88d1c4843e9b81c82a8115721b"
Sep 08 13:35:53.118151 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:53.118110    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd" event={"ID":"ca373978-0202-4f23-9f50-62660c8152f2","Type":"ContainerStarted","Data":"e1c715ed6166e69e62bf3d64c94d00f8ecbb883aad3ecd2ed7767ec0334c8faf"}
Sep 08 13:35:53.131874 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:53.131813    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-wwrkd" podStartSLOduration=2.834704334 podStartE2EDuration="6.13179228s" totalImagesPullingTime="3.297087946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:35:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:35:53.130955722 +0000 UTC m=+439.348872048" watchObservedRunningTime="2026-09-08 13:35:53.13179228 +0000 UTC m=+439.349708608"
Sep 08 13:35:55.776160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:55.776072    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:56.031416 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:56.031338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:56.132392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:56.132361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:56.652065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:56.652028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:35:57.600636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600604    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"]
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600944    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="pull"
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600956    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="pull"
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600968    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="util"
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600973    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="util"
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600979    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="extract"
Sep 08 13:35:57.601012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.600985    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="extract"
Sep 08 13:35:57.601207 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.601048    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="0224c869-4b50-4d70-853d-c1f2684dbdbb" containerName="extract"
Sep 08 13:35:57.602875 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.602859    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.604896 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.604871    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 08 13:35:57.605699 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.605668    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-hxr4v\""
Sep 08 13:35:57.605830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.605697    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 08 13:35:57.612712 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.612689    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"]
Sep 08 13:35:57.674004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.673972    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51ab330e-d91c-44f2-928a-8fe4f2adf108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.674175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.674024    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-954vh\" (UniqueName: \"kubernetes.io/projected/51ab330e-d91c-44f2-928a-8fe4f2adf108-kube-api-access-954vh\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.775464 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.775425    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51ab330e-d91c-44f2-928a-8fe4f2adf108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.775670 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.775486    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-954vh\" (UniqueName: \"kubernetes.io/projected/51ab330e-d91c-44f2-928a-8fe4f2adf108-kube-api-access-954vh\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.775890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.775867    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/51ab330e-d91c-44f2-928a-8fe4f2adf108-tmp\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.783402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.783377    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-954vh\" (UniqueName: \"kubernetes.io/projected/51ab330e-d91c-44f2-928a-8fe4f2adf108-kube-api-access-954vh\") pod \"openshift-lws-operator-fd8ccff4c-cpsp9\" (UID: \"51ab330e-d91c-44f2-928a-8fe4f2adf108\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:57.912193 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:57.912116    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"
Sep 08 13:35:58.181646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:58.181619    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9"]
Sep 08 13:35:58.183943 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:35:58.183920    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod51ab330e_d91c_44f2_928a_8fe4f2adf108.slice/crio-1978dea1b664de04c4c12965c7574b2be9258420dc8c7a4db1f80c593f747ddc WatchSource:0}: Error finding container 1978dea1b664de04c4c12965c7574b2be9258420dc8c7a4db1f80c593f747ddc: Status 404 returned error can't find the container with id 1978dea1b664de04c4c12965c7574b2be9258420dc8c7a4db1f80c593f747ddc
Sep 08 13:35:59.150203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:35:59.150171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9" event={"ID":"51ab330e-d91c-44f2-928a-8fe4f2adf108","Type":"ContainerStarted","Data":"1978dea1b664de04c4c12965c7574b2be9258420dc8c7a4db1f80c593f747ddc"}
Sep 08 13:36:00.157625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:00.157591    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9" event={"ID":"51ab330e-d91c-44f2-928a-8fe4f2adf108","Type":"ContainerStarted","Data":"1d0b4e27b15cb775ba792dd4a13a75254dd2a19ccd634785a547ef6b2678cbf4"}
Sep 08 13:36:00.171533 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:00.171446    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-cpsp9" podStartSLOduration=1.801119394 podStartE2EDuration="3.17142494s" totalImagesPullingTime="1.370305546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:35:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:00.17122749 +0000 UTC m=+446.389143840" watchObservedRunningTime="2026-09-08 13:36:00.17142494 +0000 UTC m=+446.389341265"
Sep 08 13:36:00.823092 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:00.823059    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:00.923266 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:00.923223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:02.581241 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.581208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:02.718647 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.718599    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"]
Sep 08 13:36:02.721100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.721080    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.723225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.723205    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 13:36:02.723225 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.723205    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pdttm\""
Sep 08 13:36:02.723959 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.723935    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 13:36:02.729701 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.729680    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"]
Sep 08 13:36:02.822568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.822523    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.822768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.822720    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.822768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.822748    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgvdb\" (UniqueName: \"kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.923948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.923865    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.923948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.923899    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgvdb\" (UniqueName: \"kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.923948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.923922    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.924279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.924255    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.924334 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.924294    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:02.931737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:02.931716    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgvdb\" (UniqueName: \"kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:03.030685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:03.030641    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:03.300277 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:03.300219    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"]
Sep 08 13:36:03.302356 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:03.302329    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b229bec_6122_4f76_be42_a0593bff9ca4.slice/crio-67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af WatchSource:0}: Error finding container 67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af: Status 404 returned error can't find the container with id 67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af
Sep 08 13:36:04.179954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:04.179915    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerID="ed2aa35807a1ab87e6cbacf101af1d9e43f1df036383866a1a2f647834a08aff" exitCode=0
Sep 08 13:36:04.180351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:04.180003    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p" event={"ID":"5b229bec-6122-4f76-be42-a0593bff9ca4","Type":"ContainerDied","Data":"ed2aa35807a1ab87e6cbacf101af1d9e43f1df036383866a1a2f647834a08aff"}
Sep 08 13:36:04.180351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:04.180055    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p" event={"ID":"5b229bec-6122-4f76-be42-a0593bff9ca4","Type":"ContainerStarted","Data":"67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af"}
Sep 08 13:36:05.186767 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:05.186732    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerID="8554d332e9cb2b067478e924c0fb5d342bca769c452726c11ff1a569299f4b0d" exitCode=0
Sep 08 13:36:05.187135 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:05.186792    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p" event={"ID":"5b229bec-6122-4f76-be42-a0593bff9ca4","Type":"ContainerDied","Data":"8554d332e9cb2b067478e924c0fb5d342bca769c452726c11ff1a569299f4b0d"}
Sep 08 13:36:06.193625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:06.193585    2583 generic.go:356] "Generic (PLEG): container finished" podID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerID="e4c9101125ea4686f3b991d4c5f619244df95b0a2582fb2967b3e7d6128b0972" exitCode=0
Sep 08 13:36:06.193999 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:06.193676    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p" event={"ID":"5b229bec-6122-4f76-be42-a0593bff9ca4","Type":"ContainerDied","Data":"e4c9101125ea4686f3b991d4c5f619244df95b0a2582fb2967b3e7d6128b0972"}
Sep 08 13:36:07.319536 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.319511    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:07.362882 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.362853    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle\") pod \"5b229bec-6122-4f76-be42-a0593bff9ca4\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") "
Sep 08 13:36:07.362986 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.362928    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util\") pod \"5b229bec-6122-4f76-be42-a0593bff9ca4\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") "
Sep 08 13:36:07.362986 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.362947    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hgvdb\" (UniqueName: \"kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb\") pod \"5b229bec-6122-4f76-be42-a0593bff9ca4\" (UID: \"5b229bec-6122-4f76-be42-a0593bff9ca4\") "
Sep 08 13:36:07.363584 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.363539    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle" (OuterVolumeSpecName: "bundle") pod "5b229bec-6122-4f76-be42-a0593bff9ca4" (UID: "5b229bec-6122-4f76-be42-a0593bff9ca4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:36:07.365154 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.365120    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb" (OuterVolumeSpecName: "kube-api-access-hgvdb") pod "5b229bec-6122-4f76-be42-a0593bff9ca4" (UID: "5b229bec-6122-4f76-be42-a0593bff9ca4"). InnerVolumeSpecName "kube-api-access-hgvdb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:36:07.370749 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.370710    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util" (OuterVolumeSpecName: "util") pod "5b229bec-6122-4f76-be42-a0593bff9ca4" (UID: "5b229bec-6122-4f76-be42-a0593bff9ca4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:36:07.464283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.464207    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:07.464283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.464235    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hgvdb\" (UniqueName: \"kubernetes.io/projected/5b229bec-6122-4f76-be42-a0593bff9ca4-kube-api-access-hgvdb\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:07.464283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:07.464245    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b229bec-6122-4f76-be42-a0593bff9ca4-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:08.204865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:08.204835    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p"
Sep 08 13:36:08.204865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:08.204854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afznt5p" event={"ID":"5b229bec-6122-4f76-be42-a0593bff9ca4","Type":"ContainerDied","Data":"67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af"}
Sep 08 13:36:08.205064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:08.204899    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="67f5dde4cf92e9b14ea02add17e2e114323b3129210993cc51993c26258ed1af"
Sep 08 13:36:18.998231 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998196    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"]
Sep 08 13:36:18.998638 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998622    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="extract"
Sep 08 13:36:18.998638 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998637    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="extract"
Sep 08 13:36:18.998718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998648    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="util"
Sep 08 13:36:18.998718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998654    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="util"
Sep 08 13:36:18.998718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998666    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="pull"
Sep 08 13:36:18.998718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998674    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="pull"
Sep 08 13:36:18.998842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:18.998766    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b229bec-6122-4f76-be42-a0593bff9ca4" containerName="extract"
Sep 08 13:36:19.001742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.001727    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.004521 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.004499    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 13:36:19.005267 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.005249    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 13:36:19.005327 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.005251    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pdttm\""
Sep 08 13:36:19.011965 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.011940    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"]
Sep 08 13:36:19.074144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.074107    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.074144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.074144    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.074365 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.074189    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cxjlj\" (UniqueName: \"kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.175044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.175012    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cxjlj\" (UniqueName: \"kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.175232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.175097    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.175232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.175118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.175483 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.175460    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.175596 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.175507    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.187323 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.187300    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cxjlj\" (UniqueName: \"kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj\") pod \"8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") " pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.226934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.226892    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:19.310788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.310754    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:19.412090 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.412058    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:19.597165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.597064    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"]
Sep 08 13:36:19.601601 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:19.601574    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39470a08_ae38_43d2_bd6b_bb8811526227.slice/crio-abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7 WatchSource:0}: Error finding container abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7: Status 404 returned error can't find the container with id abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7
Sep 08 13:36:19.885813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:19.885723    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:20.261723 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:20.261688    2583 generic.go:356] "Generic (PLEG): container finished" podID="39470a08-ae38-43d2-bd6b-bb8811526227" containerID="b65dafc686bcae4e1bb48840969ddfd447ddb9cf5e03e154915a8f459e230795" exitCode=0
Sep 08 13:36:20.262085 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:20.261779    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz" event={"ID":"39470a08-ae38-43d2-bd6b-bb8811526227","Type":"ContainerDied","Data":"b65dafc686bcae4e1bb48840969ddfd447ddb9cf5e03e154915a8f459e230795"}
Sep 08 13:36:20.262085 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:20.261830    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz" event={"ID":"39470a08-ae38-43d2-bd6b-bb8811526227","Type":"ContainerStarted","Data":"abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7"}
Sep 08 13:36:20.776859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:20.776822    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:20.937048 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:20.936959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:21.058132 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.058097    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"]
Sep 08 13:36:21.060492 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.060471    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.093394 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.093360    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 08 13:36:21.093588 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.093399    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 08 13:36:21.093588 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.093546    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 08 13:36:21.094218 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.094199    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 08 13:36:21.094307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.094199    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-cdfft\""
Sep 08 13:36:21.129868 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.129840    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"]
Sep 08 13:36:21.193803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.193707    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.193803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.193751    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.193803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.193776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bfrd\" (UniqueName: \"kubernetes.io/projected/1f50e301-84d5-42f9-adf3-240aa20b5d3f-kube-api-access-4bfrd\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.194032 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.193866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.194032 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.193901    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.269279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.269244    2583 generic.go:356] "Generic (PLEG): container finished" podID="39470a08-ae38-43d2-bd6b-bb8811526227" containerID="992ec9dafc74ff75e1f5495a2eaf2e418980c6740b109a58b7b1bfd1cadbe77b" exitCode=0
Sep 08 13:36:21.269713 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.269333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz" event={"ID":"39470a08-ae38-43d2-bd6b-bb8811526227","Type":"ContainerDied","Data":"992ec9dafc74ff75e1f5495a2eaf2e418980c6740b109a58b7b1bfd1cadbe77b"}
Sep 08 13:36:21.294403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294377    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.294538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294412    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.294538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294431    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4bfrd\" (UniqueName: \"kubernetes.io/projected/1f50e301-84d5-42f9-adf3-240aa20b5d3f-kube-api-access-4bfrd\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.294538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294475    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.294538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294495    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.294953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.294927    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-manifests\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.296830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.296808    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f50e301-84d5-42f9-adf3-240aa20b5d3f-tmp\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.296916 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.296904    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.296955 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.296929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1f50e301-84d5-42f9-adf3-240aa20b5d3f-webhook-cert\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.323027 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.322996    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bfrd\" (UniqueName: \"kubernetes.io/projected/1f50e301-84d5-42f9-adf3-240aa20b5d3f-kube-api-access-4bfrd\") pod \"opendatahub-operator-controller-manager-54685d959b-xc5lv\" (UID: \"1f50e301-84d5-42f9-adf3-240aa20b5d3f\") " pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.370448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.370413    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:21.677444 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.677417    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"]
Sep 08 13:36:21.679952 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:21.679898    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f50e301_84d5_42f9_adf3_240aa20b5d3f.slice/crio-2a854416a126dca5b13b44b666a434c200d2442176e75251bcdd272dccfa36d2 WatchSource:0}: Error finding container 2a854416a126dca5b13b44b666a434c200d2442176e75251bcdd272dccfa36d2: Status 404 returned error can't find the container with id 2a854416a126dca5b13b44b666a434c200d2442176e75251bcdd272dccfa36d2
Sep 08 13:36:21.864508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.864456    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:21.992208 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:21.992174    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:22.113695 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:22.113651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:22.276899 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:22.276866    2583 generic.go:356] "Generic (PLEG): container finished" podID="39470a08-ae38-43d2-bd6b-bb8811526227" containerID="840abc3689202ba9eecef295f11b3ec0ba2d22c864be9837c665f9bd9f217926" exitCode=0
Sep 08 13:36:22.277281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:22.276992    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz" event={"ID":"39470a08-ae38-43d2-bd6b-bb8811526227","Type":"ContainerDied","Data":"840abc3689202ba9eecef295f11b3ec0ba2d22c864be9837c665f9bd9f217926"}
Sep 08 13:36:22.278385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:22.278367    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv" event={"ID":"1f50e301-84d5-42f9-adf3-240aa20b5d3f","Type":"ContainerStarted","Data":"2a854416a126dca5b13b44b666a434c200d2442176e75251bcdd272dccfa36d2"}
Sep 08 13:36:23.463643 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.463515    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:23.617272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.617173    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cxjlj\" (UniqueName: \"kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj\") pod \"39470a08-ae38-43d2-bd6b-bb8811526227\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") "
Sep 08 13:36:23.617446 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.617296    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle\") pod \"39470a08-ae38-43d2-bd6b-bb8811526227\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") "
Sep 08 13:36:23.617446 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.617334    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util\") pod \"39470a08-ae38-43d2-bd6b-bb8811526227\" (UID: \"39470a08-ae38-43d2-bd6b-bb8811526227\") "
Sep 08 13:36:23.618205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.618119    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle" (OuterVolumeSpecName: "bundle") pod "39470a08-ae38-43d2-bd6b-bb8811526227" (UID: "39470a08-ae38-43d2-bd6b-bb8811526227"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:36:23.620155 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.620126    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj" (OuterVolumeSpecName: "kube-api-access-cxjlj") pod "39470a08-ae38-43d2-bd6b-bb8811526227" (UID: "39470a08-ae38-43d2-bd6b-bb8811526227"). InnerVolumeSpecName "kube-api-access-cxjlj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:36:23.624990 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.624958    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util" (OuterVolumeSpecName: "util") pod "39470a08-ae38-43d2-bd6b-bb8811526227" (UID: "39470a08-ae38-43d2-bd6b-bb8811526227"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:36:23.718431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.718395    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:23.718431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.718430    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/39470a08-ae38-43d2-bd6b-bb8811526227-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:23.718652 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:23.718445    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cxjlj\" (UniqueName: \"kubernetes.io/projected/39470a08-ae38-43d2-bd6b-bb8811526227-kube-api-access-cxjlj\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:36:24.293031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.292417    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz" event={"ID":"39470a08-ae38-43d2-bd6b-bb8811526227","Type":"ContainerDied","Data":"abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7"}
Sep 08 13:36:24.293031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.292478    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="abfffb704c8af4c2ce5ace43c2d28b9b663192e80aabd01bdac56567b86c4df7"
Sep 08 13:36:24.293031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.292436    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/8e0ea019126e0c85f070b2014e3108eef38b84ccf513200a96dc7d56b06lgrz"
Sep 08 13:36:24.892972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.892933    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f85d669c7-f7zwh"]
Sep 08 13:36:24.893430 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893415    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="pull"
Sep 08 13:36:24.893479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893433    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="pull"
Sep 08 13:36:24.893479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893474    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="util"
Sep 08 13:36:24.893586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893483    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="util"
Sep 08 13:36:24.893586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893495    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="extract"
Sep 08 13:36:24.893586 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893504    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="extract"
Sep 08 13:36:24.893705 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.893602    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="39470a08-ae38-43d2-bd6b-bb8811526227" containerName="extract"
Sep 08 13:36:24.895989 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.895972    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:24.948250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:24.948218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f85d669c7-f7zwh"]
Sep 08 13:36:25.031761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.031729    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-service-ca\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.031960 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.031776    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.031960 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.031908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-trusted-ca-bundle\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.032078 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.031959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-oauth-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.032078 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.031985    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9wcvq\" (UniqueName: \"kubernetes.io/projected/39cddde0-5706-4ced-8b0c-fbc7498123ba-kube-api-access-9wcvq\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.032078 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.032013    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.032175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.032120    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-oauth-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133222    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-service-ca\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133278    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133346    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-trusted-ca-bundle\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133490    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-oauth-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133529    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9wcvq\" (UniqueName: \"kubernetes.io/projected/39cddde0-5706-4ced-8b0c-fbc7498123ba-kube-api-access-9wcvq\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.133613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133586    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.134061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133616    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-oauth-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.134061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.133991    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-service-ca\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.134260 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.134185    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-trusted-ca-bundle\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.134315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.134270    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-oauth-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.134482 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.134461    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.136824 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.136795    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-oauth-config\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.138086 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.138062    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/39cddde0-5706-4ced-8b0c-fbc7498123ba-console-serving-cert\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.145427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.145341    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9wcvq\" (UniqueName: \"kubernetes.io/projected/39cddde0-5706-4ced-8b0c-fbc7498123ba-kube-api-access-9wcvq\") pod \"console-f85d669c7-f7zwh\" (UID: \"39cddde0-5706-4ced-8b0c-fbc7498123ba\") " pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.207356 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.207314    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:25.510361 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.510322    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:25.710319 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:25.710228    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f85d669c7-f7zwh"]
Sep 08 13:36:25.712954 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:25.712929    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39cddde0_5706_4ced_8b0c_fbc7498123ba.slice/crio-03f3bf6a3765f6df485088b61b3e50f1b1642e7194013c6358e2f536508cd233 WatchSource:0}: Error finding container 03f3bf6a3765f6df485088b61b3e50f1b1642e7194013c6358e2f536508cd233: Status 404 returned error can't find the container with id 03f3bf6a3765f6df485088b61b3e50f1b1642e7194013c6358e2f536508cd233
Sep 08 13:36:26.306735 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:26.306686    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f85d669c7-f7zwh" event={"ID":"39cddde0-5706-4ced-8b0c-fbc7498123ba","Type":"ContainerStarted","Data":"717438ca03259bd608c3d5d25439508991e0bfbd32f00207febda18280f88b0d"}
Sep 08 13:36:26.307186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:26.306759    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f85d669c7-f7zwh" event={"ID":"39cddde0-5706-4ced-8b0c-fbc7498123ba","Type":"ContainerStarted","Data":"03f3bf6a3765f6df485088b61b3e50f1b1642e7194013c6358e2f536508cd233"}
Sep 08 13:36:26.308444 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:26.308414    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv" event={"ID":"1f50e301-84d5-42f9-adf3-240aa20b5d3f","Type":"ContainerStarted","Data":"4c1879769a2e19fef4d59e3d78eced42e641debb7c26a251aaf19ecb1e0f9739"}
Sep 08 13:36:26.334286 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:26.334235    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f85d669c7-f7zwh" podStartSLOduration=2.33421911 podStartE2EDuration="2.33421911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:26.3330136 +0000 UTC m=+472.550929958" watchObservedRunningTime="2026-09-08 13:36:26.33421911 +0000 UTC m=+472.552135461"
Sep 08 13:36:26.751903 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:26.751864    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:27.314418 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:27.314386    2583 generic.go:356] "Generic (PLEG): container finished" podID="1f50e301-84d5-42f9-adf3-240aa20b5d3f" containerID="4c1879769a2e19fef4d59e3d78eced42e641debb7c26a251aaf19ecb1e0f9739" exitCode=0
Sep 08 13:36:27.314942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:27.314465    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv" event={"ID":"1f50e301-84d5-42f9-adf3-240aa20b5d3f","Type":"ContainerDied","Data":"4c1879769a2e19fef4d59e3d78eced42e641debb7c26a251aaf19ecb1e0f9739"}
Sep 08 13:36:28.322816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:28.322753    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv" event={"ID":"1f50e301-84d5-42f9-adf3-240aa20b5d3f","Type":"ContainerStarted","Data":"4d307e9ed520edf2e411249f6e8cd904f3431bfd411be9ca40fe46986b843b56"}
Sep 08 13:36:28.323299 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:28.322827    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:28.365751 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:28.365702    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv" podStartSLOduration=3.204809124 podStartE2EDuration="8.365684509s" totalImagesPullingTime="4.055850023s" totalInitContainerRuntime="1.105025362s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:20 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:28.364764617 +0000 UTC m=+474.582680942" watchObservedRunningTime="2026-09-08 13:36:28.365684509 +0000 UTC m=+474.583600834"
Sep 08 13:36:29.044503 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:29.044464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:29.149283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:29.149242    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:31.343898 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:31.343846    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:31.455642 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:31.455610    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:35.207621 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.207589    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:35.208021 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.207637    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:35.212348 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.212328    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:35.362378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.362347    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f85d669c7-f7zwh"
Sep 08 13:36:35.450498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.450463    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:36:35.611004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.610967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:35.714774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.714741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:35.813629 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:35.813594    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:39.332263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:39.332235    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-54685d959b-xc5lv"
Sep 08 13:36:42.159201 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:42.159163    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:42.375039 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:42.375003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:48.087508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.087464    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:48.100190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.100160    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"]
Sep 08 13:36:48.106221 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.106196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.112496 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.112470    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 08 13:36:48.113321 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.113305    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 08 13:36:48.113407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.113348    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 08 13:36:48.115032 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.115011    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 08 13:36:48.126744 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.126719    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"]
Sep 08 13:36:48.240532 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.240499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4890981a-15b7-47ab-b279-b26bf5e02745-tmp\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.240747 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.240635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2597x\" (UniqueName: \"kubernetes.io/projected/4890981a-15b7-47ab-b279-b26bf5e02745-kube-api-access-2597x\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.240747 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.240678    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.341916 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.341813    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.341916 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.341886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4890981a-15b7-47ab-b279-b26bf5e02745-tmp\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.342163 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:36:48.341932    2583 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 08 13:36:48.342163 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:36:48.342003    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs podName:4890981a-15b7-47ab-b279-b26bf5e02745 nodeName:}" failed. No retries permitted until 2026-09-08 13:36:48.841982444 +0000 UTC m=+495.059898760 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs") pod "kube-auth-proxy-55db547db7-6rxmr" (UID: "4890981a-15b7-47ab-b279-b26bf5e02745") : secret "kube-auth-proxy-tls" not found
Sep 08 13:36:48.342163 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.342057    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2597x\" (UniqueName: \"kubernetes.io/projected/4890981a-15b7-47ab-b279-b26bf5e02745-kube-api-access-2597x\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.344402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.344379    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4890981a-15b7-47ab-b279-b26bf5e02745-tmp\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.356918 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.356880    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2597x\" (UniqueName: \"kubernetes.io/projected/4890981a-15b7-47ab-b279-b26bf5e02745-kube-api-access-2597x\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.846339 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.846299    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.849423 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.849404    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4890981a-15b7-47ab-b279-b26bf5e02745-tls-certs\") pod \"kube-auth-proxy-55db547db7-6rxmr\" (UID: \"4890981a-15b7-47ab-b279-b26bf5e02745\") " pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:48.923874 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:48.923838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:49.017714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:49.017679    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"
Sep 08 13:36:49.344305 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:49.344269    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4890981a_15b7_47ab_b279_b26bf5e02745.slice/crio-1d0114ee486d28b39b6b1717bea6bb91b15d25d069cfea4b7cc1dbf57d1fa442 WatchSource:0}: Error finding container 1d0114ee486d28b39b6b1717bea6bb91b15d25d069cfea4b7cc1dbf57d1fa442: Status 404 returned error can't find the container with id 1d0114ee486d28b39b6b1717bea6bb91b15d25d069cfea4b7cc1dbf57d1fa442
Sep 08 13:36:49.353214 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:49.353172    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-55db547db7-6rxmr"]
Sep 08 13:36:49.431231 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:49.431196    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr" event={"ID":"4890981a-15b7-47ab-b279-b26bf5e02745","Type":"ContainerStarted","Data":"1d0114ee486d28b39b6b1717bea6bb91b15d25d069cfea4b7cc1dbf57d1fa442"}
Sep 08 13:36:50.534775 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.534741    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:36:50.536447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.536413    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:50.537482 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.537402    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.540443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.540412    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 08 13:36:50.540610 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.540416    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 13:36:50.540891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.540876    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hm24d\""
Sep 08 13:36:50.548185 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.548162    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 08 13:36:50.548308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.548167    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 08 13:36:50.565720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.565688    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:36:50.665464 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665421    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665471    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbvtq\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665503    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665769    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.665856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.665808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.675821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.675781    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:50.766748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766708    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766780    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766820    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766855    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766900    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbvtq\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.766941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.766918    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.768029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.767806    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.770035 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.769906    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.770035 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.770026    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.770213 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.770059    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.771273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.771231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.784449 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.784393    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.785665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.785602    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbvtq\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq\") pod \"istiod-openshift-gateway-7b7995d5cd-fkv5n\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:50.848626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:50.848587    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:51.613058 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:51.612974    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:36:51.615777 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:36:51.615755    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79f08867_d577_4a07_9539_e6bd190ed232.slice/crio-33ceff38e1c3ae3e3846037caab90f1c028920e04a5b3d9476845b501b3608a8 WatchSource:0}: Error finding container 33ceff38e1c3ae3e3846037caab90f1c028920e04a5b3d9476845b501b3608a8: Status 404 returned error can't find the container with id 33ceff38e1c3ae3e3846037caab90f1c028920e04a5b3d9476845b501b3608a8
Sep 08 13:36:52.448999 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:52.448966    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" event={"ID":"79f08867-d577-4a07-9539-e6bd190ed232","Type":"ContainerStarted","Data":"33ceff38e1c3ae3e3846037caab90f1c028920e04a5b3d9476845b501b3608a8"}
Sep 08 13:36:53.199853 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.199810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:53.456663 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.456585    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr" event={"ID":"4890981a-15b7-47ab-b279-b26bf5e02745","Type":"ContainerStarted","Data":"daa14830ef00eed91da729264f5c8e29306035a8210c27541a5aead20fcd1436"}
Sep 08 13:36:53.490100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.490023    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-55db547db7-6rxmr" podStartSLOduration=2.372337494 podStartE2EDuration="5.490001641s" totalImagesPullingTime="3.117664147s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:53.488533282 +0000 UTC m=+499.706449607" watchObservedRunningTime="2026-09-08 13:36:53.490001641 +0000 UTC m=+499.707917967"
Sep 08 13:36:53.684095 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.684045    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:53.860839 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.860801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:53.965051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:53.965021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:54.281312 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.281272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:54.383617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.383583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:54.425977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.425928    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:36:54.426131 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.426004    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:36:54.562452 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.562340    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:54.958275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:54.958172    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:55.298701 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.298655    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:55.375833 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.375786    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:55.472102 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.472062    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" event={"ID":"79f08867-d577-4a07-9539-e6bd190ed232","Type":"ContainerStarted","Data":"9b28f5c54769069d59f6032fd4d2d47b85a63d9d9cafbaa38cd82d16b1ae4dd0"}
Sep 08 13:36:55.472311 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.472124    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:36:55.473852 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.473828    2583 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-fkv5n container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 13:36:55.474014 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.473881    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" podUID="79f08867-d577-4a07-9539-e6bd190ed232" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:36:55.498081 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.498027    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" podStartSLOduration=2.6901042200000003 podStartE2EDuration="5.49800927s" totalImagesPullingTime="2.80790505s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:36:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:36:55.49671052 +0000 UTC m=+501.714626861" watchObservedRunningTime="2026-09-08 13:36:55.49800927 +0000 UTC m=+501.715925594"
Sep 08 13:36:55.621190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.621107    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:55.744841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:55.744803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:56.104581 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:56.104530    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:56.195455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:56.195414    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:36:56.478861 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:36:56.478778    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:37:00.470459 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.470413    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-59d7bf9f86-5jxk4" podUID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" containerName="console" containerID="cri-o://9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6" gracePeriod=15
Sep 08 13:37:00.722117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.722042    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:37:00.870431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870397    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870485    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870524    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870544    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870622    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870852 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870667    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-82v6s\" (UniqueName: \"kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.870942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870920    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle\") pod \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\" (UID: \"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57\") "
Sep 08 13:37:00.871250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.870925    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:37:00.871437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.871145    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca" (OuterVolumeSpecName: "service-ca") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:37:00.871437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.871248    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config" (OuterVolumeSpecName: "console-config") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:37:00.871437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.871258    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:37:00.872915 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.872895    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:37:00.873386 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.873360    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s" (OuterVolumeSpecName: "kube-api-access-82v6s") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "kube-api-access-82v6s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:37:00.873466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.873366    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" (UID: "ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:37:00.971997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.971954    2583 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-trusted-ca-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.971997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.971988    2583 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-oauth-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.971997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.972001    2583 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.972295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.972012    2583 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.972295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.972025    2583 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-oauth-config\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.972295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.972036    2583 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-console-serving-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:00.972295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:00.972048    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-82v6s\" (UniqueName: \"kubernetes.io/projected/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57-kube-api-access-82v6s\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:01.513485 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.513444    2583 generic.go:356] "Generic (PLEG): container finished" podID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" containerID="9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6" exitCode=0
Sep 08 13:37:01.513900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.513502    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7bf9f86-5jxk4" event={"ID":"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57","Type":"ContainerDied","Data":"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"}
Sep 08 13:37:01.513900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.513502    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59d7bf9f86-5jxk4"
Sep 08 13:37:01.513900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.513597    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59d7bf9f86-5jxk4" event={"ID":"ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57","Type":"ContainerDied","Data":"09a4dab78268c7747f1c84360520ed161abfadcdb614169f65aae68b230a62b5"}
Sep 08 13:37:01.513900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.513625    2583 scope.go:117] "RemoveContainer" containerID="9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"
Sep 08 13:37:01.522828 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.522809    2583 scope.go:117] "RemoveContainer" containerID="9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"
Sep 08 13:37:01.523182 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:37:01.523159    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6\": container with ID starting with 9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6 not found: ID does not exist" containerID="9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"
Sep 08 13:37:01.523252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.523196    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6"} err="failed to get container status \"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6\": rpc error: code = NotFound desc = could not find container \"9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6\": container with ID starting with 9371fe1621950df358b8376c5bbc77099da6fdc68fe023072dfc68c064c3dfb6 not found: ID does not exist"
Sep 08 13:37:01.581423 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.581380    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:37:01.590793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:01.590758    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59d7bf9f86-5jxk4"]
Sep 08 13:37:02.231320 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:02.231285    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" path="/var/lib/kubelet/pods/ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57/volumes"
Sep 08 13:37:06.352329 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.352293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:06.487299 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.487266    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"]
Sep 08 13:37:06.487665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.487652    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" containerName="console"
Sep 08 13:37:06.487721 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.487668    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" containerName="console"
Sep 08 13:37:06.487754 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.487725    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff3f88ae-4cc5-4424-ba4a-11c2c37d1c57" containerName="console"
Sep 08 13:37:06.491957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.491942    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.494894 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.494875    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 08 13:37:06.495355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.495341    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-kzl5z\""
Sep 08 13:37:06.513195 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.513164    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"]
Sep 08 13:37:06.517012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.516983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"]
Sep 08 13:37:06.517138 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.517082    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.525753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.525726    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 08 13:37:06.525753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.525748    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-27hcn\""
Sep 08 13:37:06.550005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.549971    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"]
Sep 08 13:37:06.627355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.627265    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmhxq\" (UniqueName: \"kubernetes.io/projected/3e84895b-aa10-40f5-8040-abd64742ec1e-kube-api-access-lmhxq\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.627355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.627332    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e84895b-aa10-40f5-8040-abd64742ec1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.627545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.627391    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqjcg\" (UniqueName: \"kubernetes.io/projected/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-kube-api-access-pqjcg\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.627545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.627407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.728240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.728201    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e84895b-aa10-40f5-8040-abd64742ec1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.728407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.728265    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqjcg\" (UniqueName: \"kubernetes.io/projected/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-kube-api-access-pqjcg\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.728407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.728283    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.728407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.728332    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmhxq\" (UniqueName: \"kubernetes.io/projected/3e84895b-aa10-40f5-8040-abd64742ec1e-kube-api-access-lmhxq\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.730976 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.730945    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.731087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.730990    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3e84895b-aa10-40f5-8040-abd64742ec1e-cert\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.745397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.745368    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqjcg\" (UniqueName: \"kubernetes.io/projected/99c1fe38-ecb8-4c64-a0e1-f039b4650ae7-kube-api-access-pqjcg\") pod \"llmisvc-controller-manager-777c6bd7d5-lwsx7\" (UID: \"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:06.754611 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.754584    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmhxq\" (UniqueName: \"kubernetes.io/projected/3e84895b-aa10-40f5-8040-abd64742ec1e-kube-api-access-lmhxq\") pod \"kserve-controller-manager-d6f4b65b8-x2vn9\" (UID: \"3e84895b-aa10-40f5-8040-abd64742ec1e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.802510 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.802472    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:06.829579 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:06.826968    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:07.044183 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.044147    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:07.131175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.131138    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"]
Sep 08 13:37:07.136001 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:07.135976    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e84895b_aa10_40f5_8040_abd64742ec1e.slice/crio-0429f97adbb39a1af1e6ba6ac3d03e080c09ac540fce87f571533a31aef12170 WatchSource:0}: Error finding container 0429f97adbb39a1af1e6ba6ac3d03e080c09ac540fce87f571533a31aef12170: Status 404 returned error can't find the container with id 0429f97adbb39a1af1e6ba6ac3d03e080c09ac540fce87f571533a31aef12170
Sep 08 13:37:07.178495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.178472    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"]
Sep 08 13:37:07.180064 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:07.180035    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod99c1fe38_ecb8_4c64_a0e1_f039b4650ae7.slice/crio-b184b9af01f0e22ba71c3545d5c8be8180a97962c8fa05e89ee9b9b853518ae8 WatchSource:0}: Error finding container b184b9af01f0e22ba71c3545d5c8be8180a97962c8fa05e89ee9b9b853518ae8: Status 404 returned error can't find the container with id b184b9af01f0e22ba71c3545d5c8be8180a97962c8fa05e89ee9b9b853518ae8
Sep 08 13:37:07.211045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.211013    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:07.403996 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.403898    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:37:07.404494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.404124    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" podUID="79f08867-d577-4a07-9539-e6bd190ed232" containerName="discovery" containerID="cri-o://9b28f5c54769069d59f6032fd4d2d47b85a63d9d9cafbaa38cd82d16b1ae4dd0" gracePeriod=30
Sep 08 13:37:07.553330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.553296    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9" event={"ID":"3e84895b-aa10-40f5-8040-abd64742ec1e","Type":"ContainerStarted","Data":"0429f97adbb39a1af1e6ba6ac3d03e080c09ac540fce87f571533a31aef12170"}
Sep 08 13:37:07.555112 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.555086    2583 generic.go:356] "Generic (PLEG): container finished" podID="79f08867-d577-4a07-9539-e6bd190ed232" containerID="9b28f5c54769069d59f6032fd4d2d47b85a63d9d9cafbaa38cd82d16b1ae4dd0" exitCode=0
Sep 08 13:37:07.555271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.555154    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" event={"ID":"79f08867-d577-4a07-9539-e6bd190ed232","Type":"ContainerDied","Data":"9b28f5c54769069d59f6032fd4d2d47b85a63d9d9cafbaa38cd82d16b1ae4dd0"}
Sep 08 13:37:07.556565 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.556526    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7" event={"ID":"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7","Type":"ContainerStarted","Data":"b184b9af01f0e22ba71c3545d5c8be8180a97962c8fa05e89ee9b9b853518ae8"}
Sep 08 13:37:07.647746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.647714    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:37:07.781917 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.781879    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:07.839902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.839869    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.839902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.839919    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bbvtq\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.840160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.840002    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.840160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.840034    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.840160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.840137    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.840160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.840159    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.840365 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.840192    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert\") pod \"79f08867-d577-4a07-9539-e6bd190ed232\" (UID: \"79f08867-d577-4a07-9539-e6bd190ed232\") "
Sep 08 13:37:07.841728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.841657    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:37:07.847876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.847818    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs" (OuterVolumeSpecName: "local-certs") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:37:07.848015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.847978    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts" (OuterVolumeSpecName: "cacerts") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:37:07.848107 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.848070    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:37:07.849830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.849787    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:37:07.849830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.849807    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token" (OuterVolumeSpecName: "istio-token") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:37:07.850423 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.850380    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq" (OuterVolumeSpecName: "kube-api-access-bbvtq") pod "79f08867-d577-4a07-9539-e6bd190ed232" (UID: "79f08867-d577-4a07-9539-e6bd190ed232"). InnerVolumeSpecName "kube-api-access-bbvtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941267    2583 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-kubeconfig\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941304    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-istio-token\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941317    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-dns-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941330    2583 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/79f08867-d577-4a07-9539-e6bd190ed232-cacerts\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941344    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bbvtq\" (UniqueName: \"kubernetes.io/projected/79f08867-d577-4a07-9539-e6bd190ed232-kube-api-access-bbvtq\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941358    2583 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/79f08867-d577-4a07-9539-e6bd190ed232-local-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:07.941398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:07.941372    2583 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/79f08867-d577-4a07-9539-e6bd190ed232-istio-csr-ca-configmap\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:37:08.565073 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:08.565041    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"
Sep 08 13:37:08.565073 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:08.565063    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n" event={"ID":"79f08867-d577-4a07-9539-e6bd190ed232","Type":"ContainerDied","Data":"33ceff38e1c3ae3e3846037caab90f1c028920e04a5b3d9476845b501b3608a8"}
Sep 08 13:37:08.565626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:08.565122    2583 scope.go:117] "RemoveContainer" containerID="9b28f5c54769069d59f6032fd4d2d47b85a63d9d9cafbaa38cd82d16b1ae4dd0"
Sep 08 13:37:08.623605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:08.623567    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:37:08.627450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:08.627415    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-fkv5n"]
Sep 08 13:37:09.276836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:09.276801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:09.400725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:09.400683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:10.230749 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.230668    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79f08867-d577-4a07-9539-e6bd190ed232" path="/var/lib/kubelet/pods/79f08867-d577-4a07-9539-e6bd190ed232/volumes"
Sep 08 13:37:10.577860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.577821    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7" event={"ID":"99c1fe38-ecb8-4c64-a0e1-f039b4650ae7","Type":"ContainerStarted","Data":"39cee9ef6e21edd6ea7367283f6d577467c1b7d6e3b06116ca30dc403080ca33"}
Sep 08 13:37:10.578033 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.577910    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:10.579566 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.579528    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9" event={"ID":"3e84895b-aa10-40f5-8040-abd64742ec1e","Type":"ContainerStarted","Data":"074458dc9e8c2ca717560dc6d55743b0da4e52e194f287f71d25c719dfd91c17"}
Sep 08 13:37:10.579713 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.579623    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:10.631597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.631520    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7" podStartSLOduration=1.869283845 podStartE2EDuration="4.631502336s" totalImagesPullingTime="2.762218491s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:37:10.630982373 +0000 UTC m=+516.848898699" watchObservedRunningTime="2026-09-08 13:37:10.631502336 +0000 UTC m=+516.849418661"
Sep 08 13:37:10.731134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:10.731067    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9" podStartSLOduration=1.922453998 podStartE2EDuration="4.731042755s" totalImagesPullingTime="2.808588757s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:37:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:37:10.726272433 +0000 UTC m=+516.944188785" watchObservedRunningTime="2026-09-08 13:37:10.731042755 +0000 UTC m=+516.948959083"
Sep 08 13:37:21.165190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:21.165148    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:21.270093 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:21.270049    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:41.587480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:41.587448    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-lwsx7"
Sep 08 13:37:41.590574 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:41.590535    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-x2vn9"
Sep 08 13:37:54.535757 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:54.535722    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:54.603384 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:54.603319    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:37:58.231500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.231471    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"]
Sep 08 13:37:58.231879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.231827    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79f08867-d577-4a07-9539-e6bd190ed232" containerName="discovery"
Sep 08 13:37:58.231879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.231839    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="79f08867-d577-4a07-9539-e6bd190ed232" containerName="discovery"
Sep 08 13:37:58.231951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.231897    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="79f08867-d577-4a07-9539-e6bd190ed232" containerName="discovery"
Sep 08 13:37:58.235064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.235049    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.237362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.237341    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 08 13:37:58.237362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.237354    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 08 13:37:58.238294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.238277    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-pdttm\""
Sep 08 13:37:58.243682 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.243660    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"]
Sep 08 13:37:58.327391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.327355    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"]
Sep 08 13:37:58.330973 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.330957    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.339833 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.339809    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"]
Sep 08 13:37:58.388861 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.388820    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.389057 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.388908    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmsxt\" (UniqueName: \"kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.389057 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.389028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.435251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.435212    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"]
Sep 08 13:37:58.439019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.438995    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.448522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.448494    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"]
Sep 08 13:37:58.490144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.490144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490108    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.490144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490141    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.490383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmsxt\" (UniqueName: \"kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.490383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490317    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dx2z\" (UniqueName: \"kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.490383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490359    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.490519 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490498    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.490585 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.490518    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.498261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.498242    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmsxt\" (UniqueName: \"kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt\") pod \"aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") " pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.532177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.532147    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"]
Sep 08 13:37:58.536314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.536292    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.544946 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.544920    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"]
Sep 08 13:37:58.545055 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.545022    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:37:58.591428 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591389    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.591626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591462    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpscj\" (UniqueName: \"kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.591626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591507    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.591626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591584    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.592196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591639    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.592196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591685    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dx2z\" (UniqueName: \"kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.592196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.591816    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.592196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.592089    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.601481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.601459    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dx2z\" (UniqueName: \"kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z\") pod \"cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") " pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.641101 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.641065    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:37:58.693177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693138    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpscj\" (UniqueName: \"kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.693366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693193    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.693448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693367    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.693512 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693478    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.693582 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693539    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxctt\" (UniqueName: \"kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.693641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693598    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.694071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693841    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.694071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.693955    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.703436 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.703346    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpscj\" (UniqueName: \"kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj\") pod \"71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") " pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.748914 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.748836    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:37:58.794265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.794228    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.794668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.794637    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.794840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.794764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxctt\" (UniqueName: \"kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.794840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.794805    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.795247 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.795140    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.818467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.818443    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxctt\" (UniqueName: \"kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt\") pod \"eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") " pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.834923 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.834895    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"]
Sep 08 13:37:58.835058 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:58.834897    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4624485_3f76_42f7_8b1b_ff18f6ce5fd3.slice/crio-4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69 WatchSource:0}: Error finding container 4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69: Status 404 returned error can't find the container with id 4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69
Sep 08 13:37:58.846351 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.846323    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:37:58.928983 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:58.928828    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"]
Sep 08 13:37:58.929110 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:58.928927    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70874a20_3110_4bf1_83a0_7e1d53f0c1e7.slice/crio-585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02 WatchSource:0}: Error finding container 585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02: Status 404 returned error can't find the container with id 585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02
Sep 08 13:37:59.027082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.027058    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"]
Sep 08 13:37:59.028821 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:59.028791    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11edffae_65c8_468c_be33_076fe975656d.slice/crio-b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393 WatchSource:0}: Error finding container b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393: Status 404 returned error can't find the container with id b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393
Sep 08 13:37:59.129619 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.129592    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"]
Sep 08 13:37:59.137594 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:37:59.137543    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89ce304b_9969_48af_8a61_a01e7c97d82f.slice/crio-42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6 WatchSource:0}: Error finding container 42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6: Status 404 returned error can't find the container with id 42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6
Sep 08 13:37:59.827706 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.827669    2583 generic.go:356] "Generic (PLEG): container finished" podID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerID="c5d4e3b05258e095cb42da36825d7d844a4a0eb2323ac26e864f84f16212c07a" exitCode=0
Sep 08 13:37:59.828171 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.827755    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5" event={"ID":"70874a20-3110-4bf1-83a0-7e1d53f0c1e7","Type":"ContainerDied","Data":"c5d4e3b05258e095cb42da36825d7d844a4a0eb2323ac26e864f84f16212c07a"}
Sep 08 13:37:59.828171 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.827804    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5" event={"ID":"70874a20-3110-4bf1-83a0-7e1d53f0c1e7","Type":"ContainerStarted","Data":"585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02"}
Sep 08 13:37:59.829472 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.829450    2583 generic.go:356] "Generic (PLEG): container finished" podID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerID="175f84829a4e2b3edd29aa95f8af1c926baa15acfe52a7336b7f6054e1f176ed" exitCode=0
Sep 08 13:37:59.829668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.829645    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr" event={"ID":"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3","Type":"ContainerDied","Data":"175f84829a4e2b3edd29aa95f8af1c926baa15acfe52a7336b7f6054e1f176ed"}
Sep 08 13:37:59.829762 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.829695    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr" event={"ID":"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3","Type":"ContainerStarted","Data":"4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69"}
Sep 08 13:37:59.831251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.831233    2583 generic.go:356] "Generic (PLEG): container finished" podID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerID="b176da09552bea4a34a409207b5874185cac09d733c9a5180e59627c6166f3f1" exitCode=0
Sep 08 13:37:59.831342 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.831302    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx" event={"ID":"89ce304b-9969-48af-8a61-a01e7c97d82f","Type":"ContainerDied","Data":"b176da09552bea4a34a409207b5874185cac09d733c9a5180e59627c6166f3f1"}
Sep 08 13:37:59.831342 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.831338    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx" event={"ID":"89ce304b-9969-48af-8a61-a01e7c97d82f","Type":"ContainerStarted","Data":"42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6"}
Sep 08 13:37:59.833114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.833094    2583 generic.go:356] "Generic (PLEG): container finished" podID="11edffae-65c8-468c-be33-076fe975656d" containerID="d726e5ee4df91b15ab388a23cf50d5e7497dce4f6ebb4ffb5661c25be89666aa" exitCode=0
Sep 08 13:37:59.833190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.833122    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s" event={"ID":"11edffae-65c8-468c-be33-076fe975656d","Type":"ContainerDied","Data":"d726e5ee4df91b15ab388a23cf50d5e7497dce4f6ebb4ffb5661c25be89666aa"}
Sep 08 13:37:59.833190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:37:59.833148    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s" event={"ID":"11edffae-65c8-468c-be33-076fe975656d","Type":"ContainerStarted","Data":"b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393"}
Sep 08 13:38:00.840812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.840779    2583 generic.go:356] "Generic (PLEG): container finished" podID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerID="b935199d616932579de4dad116fa2443878593132f8aa1a09e463bca4f6f567b" exitCode=0
Sep 08 13:38:00.841230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.840867    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5" event={"ID":"70874a20-3110-4bf1-83a0-7e1d53f0c1e7","Type":"ContainerDied","Data":"b935199d616932579de4dad116fa2443878593132f8aa1a09e463bca4f6f567b"}
Sep 08 13:38:00.842900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.842831    2583 generic.go:356] "Generic (PLEG): container finished" podID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerID="9c18255417cbd34505371a1ca61a0d0edd81c6a6c7fa06cceb5ebf94fd8b4fd8" exitCode=0
Sep 08 13:38:00.842988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.842915    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr" event={"ID":"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3","Type":"ContainerDied","Data":"9c18255417cbd34505371a1ca61a0d0edd81c6a6c7fa06cceb5ebf94fd8b4fd8"}
Sep 08 13:38:00.844946 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.844926    2583 generic.go:356] "Generic (PLEG): container finished" podID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerID="368d7232a4c7c9ec2a6cf200e0d10024acfffefea25f0a96ae0539e090947aa8" exitCode=0
Sep 08 13:38:00.845044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.844991    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx" event={"ID":"89ce304b-9969-48af-8a61-a01e7c97d82f","Type":"ContainerDied","Data":"368d7232a4c7c9ec2a6cf200e0d10024acfffefea25f0a96ae0539e090947aa8"}
Sep 08 13:38:00.847117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.847095    2583 generic.go:356] "Generic (PLEG): container finished" podID="11edffae-65c8-468c-be33-076fe975656d" containerID="0150c04ed32fe2ce885d1b63282d60ca08697c336422c7a324b2c606b84fb936" exitCode=0
Sep 08 13:38:00.847199 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:00.847155    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s" event={"ID":"11edffae-65c8-468c-be33-076fe975656d","Type":"ContainerDied","Data":"0150c04ed32fe2ce885d1b63282d60ca08697c336422c7a324b2c606b84fb936"}
Sep 08 13:38:01.854537 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.854447    2583 generic.go:356] "Generic (PLEG): container finished" podID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerID="4e933b88f020598090d76fe1e1a471603701a81d8455f62c9a41cbeca8c1d1bc" exitCode=0
Sep 08 13:38:01.854949 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.854539    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5" event={"ID":"70874a20-3110-4bf1-83a0-7e1d53f0c1e7","Type":"ContainerDied","Data":"4e933b88f020598090d76fe1e1a471603701a81d8455f62c9a41cbeca8c1d1bc"}
Sep 08 13:38:01.856721 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.856699    2583 generic.go:356] "Generic (PLEG): container finished" podID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerID="32db8b73961583417f537bbb0d762348b7201fcaa7fe473de26cadb08aa969dc" exitCode=0
Sep 08 13:38:01.856845 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.856725    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr" event={"ID":"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3","Type":"ContainerDied","Data":"32db8b73961583417f537bbb0d762348b7201fcaa7fe473de26cadb08aa969dc"}
Sep 08 13:38:01.858778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.858760    2583 generic.go:356] "Generic (PLEG): container finished" podID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerID="b0a9555b5cfabf615c9775e465b5a36e8048c663870c263c0339db99f49f3b91" exitCode=0
Sep 08 13:38:01.858865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.858781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx" event={"ID":"89ce304b-9969-48af-8a61-a01e7c97d82f","Type":"ContainerDied","Data":"b0a9555b5cfabf615c9775e465b5a36e8048c663870c263c0339db99f49f3b91"}
Sep 08 13:38:01.860817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.860801    2583 generic.go:356] "Generic (PLEG): container finished" podID="11edffae-65c8-468c-be33-076fe975656d" containerID="ad6dc45f11ca3348189a1d8f3a0c6109fbb197f8d5c97908a739474ced7126a4" exitCode=0
Sep 08 13:38:01.860884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:01.860873    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s" event={"ID":"11edffae-65c8-468c-be33-076fe975656d","Type":"ContainerDied","Data":"ad6dc45f11ca3348189a1d8f3a0c6109fbb197f8d5c97908a739474ced7126a4"}
Sep 08 13:38:03.056402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.056379    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:38:03.060105 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.060081    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:38:03.064090 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.064069    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:38:03.069815 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.069799    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:38:03.132152 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132072    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmsxt\" (UniqueName: \"kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt\") pod \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") "
Sep 08 13:38:03.132152 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132119    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dx2z\" (UniqueName: \"kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z\") pod \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") "
Sep 08 13:38:03.132379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132156    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxctt\" (UniqueName: \"kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt\") pod \"89ce304b-9969-48af-8a61-a01e7c97d82f\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") "
Sep 08 13:38:03.132379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132183    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle\") pod \"89ce304b-9969-48af-8a61-a01e7c97d82f\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") "
Sep 08 13:38:03.132379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132239    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle\") pod \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") "
Sep 08 13:38:03.132379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132315    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle\") pod \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") "
Sep 08 13:38:03.132379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132360    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util\") pod \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\" (UID: \"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3\") "
Sep 08 13:38:03.132660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132412    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle\") pod \"11edffae-65c8-468c-be33-076fe975656d\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") "
Sep 08 13:38:03.132660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132444    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util\") pod \"11edffae-65c8-468c-be33-076fe975656d\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") "
Sep 08 13:38:03.132660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132476    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zpscj\" (UniqueName: \"kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj\") pod \"11edffae-65c8-468c-be33-076fe975656d\" (UID: \"11edffae-65c8-468c-be33-076fe975656d\") "
Sep 08 13:38:03.132660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132524    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util\") pod \"89ce304b-9969-48af-8a61-a01e7c97d82f\" (UID: \"89ce304b-9969-48af-8a61-a01e7c97d82f\") "
Sep 08 13:38:03.133190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.132981    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle" (OuterVolumeSpecName: "bundle") pod "70874a20-3110-4bf1-83a0-7e1d53f0c1e7" (UID: "70874a20-3110-4bf1-83a0-7e1d53f0c1e7"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.135493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.133434    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle" (OuterVolumeSpecName: "bundle") pod "11edffae-65c8-468c-be33-076fe975656d" (UID: "11edffae-65c8-468c-be33-076fe975656d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.135493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.133798    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle" (OuterVolumeSpecName: "bundle") pod "89ce304b-9969-48af-8a61-a01e7c97d82f" (UID: "89ce304b-9969-48af-8a61-a01e7c97d82f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.135493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.134942    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle" (OuterVolumeSpecName: "bundle") pod "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" (UID: "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.135493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.135419    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z" (OuterVolumeSpecName: "kube-api-access-2dx2z") pod "70874a20-3110-4bf1-83a0-7e1d53f0c1e7" (UID: "70874a20-3110-4bf1-83a0-7e1d53f0c1e7"). InnerVolumeSpecName "kube-api-access-2dx2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:03.135493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.135425    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt" (OuterVolumeSpecName: "kube-api-access-qmsxt") pod "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" (UID: "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3"). InnerVolumeSpecName "kube-api-access-qmsxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:03.135843 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.135510    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt" (OuterVolumeSpecName: "kube-api-access-zxctt") pod "89ce304b-9969-48af-8a61-a01e7c97d82f" (UID: "89ce304b-9969-48af-8a61-a01e7c97d82f"). InnerVolumeSpecName "kube-api-access-zxctt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:03.136129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.136113    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj" (OuterVolumeSpecName: "kube-api-access-zpscj") pod "11edffae-65c8-468c-be33-076fe975656d" (UID: "11edffae-65c8-468c-be33-076fe975656d"). InnerVolumeSpecName "kube-api-access-zpscj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:03.140609 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.140538    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util" (OuterVolumeSpecName: "util") pod "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" (UID: "a4624485-3f76-42f7-8b1b-ff18f6ce5fd3"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.141412 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.141370    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util" (OuterVolumeSpecName: "util") pod "11edffae-65c8-468c-be33-076fe975656d" (UID: "11edffae-65c8-468c-be33-076fe975656d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.142325 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.142298    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util" (OuterVolumeSpecName: "util") pod "89ce304b-9969-48af-8a61-a01e7c97d82f" (UID: "89ce304b-9969-48af-8a61-a01e7c97d82f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.233546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233509    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util\") pod \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\" (UID: \"70874a20-3110-4bf1-83a0-7e1d53f0c1e7\") "
Sep 08 13:38:03.233746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233735    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233749    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11edffae-65c8-468c-be33-076fe975656d-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233757    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpscj\" (UniqueName: \"kubernetes.io/projected/11edffae-65c8-468c-be33-076fe975656d-kube-api-access-zpscj\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233766    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233775    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmsxt\" (UniqueName: \"kubernetes.io/projected/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-kube-api-access-qmsxt\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233782    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dx2z\" (UniqueName: \"kubernetes.io/projected/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-kube-api-access-2dx2z\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233791    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxctt\" (UniqueName: \"kubernetes.io/projected/89ce304b-9969-48af-8a61-a01e7c97d82f-kube-api-access-zxctt\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233799    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/89ce304b-9969-48af-8a61-a01e7c97d82f-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233807    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233815    2583 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-bundle\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.233953 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.233822    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/a4624485-3f76-42f7-8b1b-ff18f6ce5fd3-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.238880 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.238853    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util" (OuterVolumeSpecName: "util") pod "70874a20-3110-4bf1-83a0-7e1d53f0c1e7" (UID: "70874a20-3110-4bf1-83a0-7e1d53f0c1e7"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:03.334792 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.334759    2583 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/70874a20-3110-4bf1-83a0-7e1d53f0c1e7-util\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:03.874573 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.874519    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx" event={"ID":"89ce304b-9969-48af-8a61-a01e7c97d82f","Type":"ContainerDied","Data":"42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6"}
Sep 08 13:38:03.874750 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.874587    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="42197db73f564741f8eb2d70e56814d5d6e4cf9e81ba70387bcd9f871e30b6b6"
Sep 08 13:38:03.874750 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.874536    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/eb753013ef17676ca428dee58520cdfd0d566802382f786ec048d87053kq8rx"
Sep 08 13:38:03.876528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.876508    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s"
Sep 08 13:38:03.876670 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.876503    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/71cfa9a46429353ffda10db9087bedfaf65d54e1add7115bd006bed3a0b9j2s" event={"ID":"11edffae-65c8-468c-be33-076fe975656d","Type":"ContainerDied","Data":"b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393"}
Sep 08 13:38:03.876670 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.876621    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b537596480c22fbcad9f9fea871015b66bd6dbc878b9722559c686d3984a4393"
Sep 08 13:38:03.878520 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.878497    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5" event={"ID":"70874a20-3110-4bf1-83a0-7e1d53f0c1e7","Type":"ContainerDied","Data":"585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02"}
Sep 08 13:38:03.878658 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.878535    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="585681f19c391938be5395db5ebf6eb377428f4a5851be5c4cac03756819cd02"
Sep 08 13:38:03.878658 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.878504    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/cd2c7243cb31b10fd1249163c10d5ea652384acf3e15d0664e86d7183cnqcq5"
Sep 08 13:38:03.880621 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.880604    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr"
Sep 08 13:38:03.880710 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.880604    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/aa4484d327db8038362a08ac827b2b7d95fdfb52c243a86a3fcc5d78bbl2vxr" event={"ID":"a4624485-3f76-42f7-8b1b-ff18f6ce5fd3","Type":"ContainerDied","Data":"4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69"}
Sep 08 13:38:03.880750 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:03.880734    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4e4a4de5633b4e979e93098ae0dca4208cb5e8d970b14d425956961a9dddbe69"
Sep 08 13:38:07.133933 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:07.133897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:08.447619 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447586    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"]
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447943    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447954    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447961    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447967    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447975    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447980    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447990    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.447995    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448002    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448006    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="pull"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448017    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448022    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="extract"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448030    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="util"
Sep 08 13:38:08.448029 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448035    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448042    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448048    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448056    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="pull"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448061    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="pull"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448066    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="extract"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448071    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="extract"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448077    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448081    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448091    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448096    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="util"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448146    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="89ce304b-9969-48af-8a61-a01e7c97d82f" containerName="extract"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448156    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="70874a20-3110-4bf1-83a0-7e1d53f0c1e7" containerName="extract"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448162    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="11edffae-65c8-468c-be33-076fe975656d" containerName="extract"
Sep 08 13:38:08.448466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.448169    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a4624485-3f76-42f7-8b1b-ff18f6ce5fd3" containerName="extract"
Sep 08 13:38:08.452490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.452474    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:08.455381 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.455358    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 08 13:38:08.456227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.456206    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-m669g\""
Sep 08 13:38:08.456304 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.456206    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 08 13:38:08.464514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.464491    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"]
Sep 08 13:38:08.486104 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.486082    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h9wpr\" (UniqueName: \"kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr\") pod \"limitador-operator-controller-manager-6c549b6d85-xl55b\" (UID: \"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:08.587392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.587341    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h9wpr\" (UniqueName: \"kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr\") pod \"limitador-operator-controller-manager-6c549b6d85-xl55b\" (UID: \"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:08.598319 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.598286    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h9wpr\" (UniqueName: \"kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr\") pod \"limitador-operator-controller-manager-6c549b6d85-xl55b\" (UID: \"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:08.763446 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:08.763362    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:09.041905 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:09.041873    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"]
Sep 08 13:38:09.045395 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:09.045370    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod296ece16_a4ea_40b0_acdc_c2c0dbf56ac4.slice/crio-e0a446d32ea20dea7978335f1e41a921ad97d84b1e64870d72e265bfa8dc8366 WatchSource:0}: Error finding container e0a446d32ea20dea7978335f1e41a921ad97d84b1e64870d72e265bfa8dc8366: Status 404 returned error can't find the container with id e0a446d32ea20dea7978335f1e41a921ad97d84b1e64870d72e265bfa8dc8366
Sep 08 13:38:09.201783 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:09.201741    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:09.918893 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:09.918853    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b" event={"ID":"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4","Type":"ContainerStarted","Data":"e0a446d32ea20dea7978335f1e41a921ad97d84b1e64870d72e265bfa8dc8366"}
Sep 08 13:38:10.504082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.504050    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"]
Sep 08 13:38:10.507544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.507522    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:10.509777 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:10.509745    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"dns-operator-controller-manager-5985564776-lg74b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="9cd612c8-ddb4-4354-a95b-14621950f461" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:10.509903 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:10.509882    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"dns-operator-controller-manager-dockercfg-c49kk\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c49kk\"" type="*v1.Secret"
Sep 08 13:38:10.510376 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:10.510352    2583 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"dns-operator-controller-env\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" type="*v1.ConfigMap"
Sep 08 13:38:10.530659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.530624    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"]
Sep 08 13:38:10.608040 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.608006    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcbxw\" (UniqueName: \"kubernetes.io/projected/9cd612c8-ddb4-4354-a95b-14621950f461-kube-api-access-rcbxw\") pod \"dns-operator-controller-manager-5985564776-lg74b\" (UID: \"9cd612c8-ddb4-4354-a95b-14621950f461\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:10.709543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.709508    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcbxw\" (UniqueName: \"kubernetes.io/projected/9cd612c8-ddb4-4354-a95b-14621950f461-kube-api-access-rcbxw\") pod \"dns-operator-controller-manager-5985564776-lg74b\" (UID: \"9cd612c8-ddb4-4354-a95b-14621950f461\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:10.744843 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.744811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcbxw\" (UniqueName: \"kubernetes.io/projected/9cd612c8-ddb4-4354-a95b-14621950f461-kube-api-access-rcbxw\") pod \"dns-operator-controller-manager-5985564776-lg74b\" (UID: \"9cd612c8-ddb4-4354-a95b-14621950f461\") " pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:10.981830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:10.981794    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:11.387774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.387740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:11.757752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.757664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:11.817264 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.817226    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 08 13:38:11.818994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.818963    2583 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b" secret="" err="failed to sync secret cache: timed out waiting for the condition"
Sep 08 13:38:11.819111 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.819043    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:11.857734 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:11.857691    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:12.057862 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.057834    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-c49kk\""
Sep 08 13:38:12.110141 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.110117    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"]
Sep 08 13:38:12.111323 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:12.111303    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cd612c8_ddb4_4354_a95b_14621950f461.slice/crio-ce4c9004600c172d87d33e5cb121ac67db934f6fc99cbb69ead075fe356e5b82 WatchSource:0}: Error finding container ce4c9004600c172d87d33e5cb121ac67db934f6fc99cbb69ead075fe356e5b82: Status 404 returned error can't find the container with id ce4c9004600c172d87d33e5cb121ac67db934f6fc99cbb69ead075fe356e5b82
Sep 08 13:38:12.826678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.826640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:12.948083 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.948046    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b" event={"ID":"9cd612c8-ddb4-4354-a95b-14621950f461","Type":"ContainerStarted","Data":"ce4c9004600c172d87d33e5cb121ac67db934f6fc99cbb69ead075fe356e5b82"}
Sep 08 13:38:12.950092 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.950068    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b" event={"ID":"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4","Type":"ContainerStarted","Data":"4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d"}
Sep 08 13:38:12.950206 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.950163    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:12.986814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:12.986596    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b" podStartSLOduration=2.124577884 podStartE2EDuration="4.986575975s" totalImagesPullingTime="2.861998091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:12.98557823 +0000 UTC m=+579.203494578" watchObservedRunningTime="2026-09-08 13:38:12.986575975 +0000 UTC m=+579.204492296"
Sep 08 13:38:13.960575 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:13.960497    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b" event={"ID":"9cd612c8-ddb4-4354-a95b-14621950f461","Type":"ContainerStarted","Data":"d7db085cb8a9e157e2c757ebc921fc9284fcb9b6f3c092db7e4f2ea824477d56"}
Sep 08 13:38:13.961112 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:13.960805    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:13.962344 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:13.962314    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:14.010865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:14.010803    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b" podStartSLOduration=2.665235713 podStartE2EDuration="4.010782969s" totalImagesPullingTime="1.345547256s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:14.010583251 +0000 UTC m=+580.228499597" watchObservedRunningTime="2026-09-08 13:38:14.010782969 +0000 UTC m=+580.228699294"
Sep 08 13:38:15.229945 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.229915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:15.325562 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.325528    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"]
Sep 08 13:38:15.329107 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.329090    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.331437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.331423    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rdd2z\""
Sep 08 13:38:15.344567 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.344535    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"]
Sep 08 13:38:15.422506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.422472    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:15.460657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.460624    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2l9kc\" (UniqueName: \"kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.460814 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.460675    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.561906 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.561869    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2l9kc\" (UniqueName: \"kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.562072 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.561943    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.562305 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.562286    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.585813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.585779    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:15.588176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.588157    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2l9kc\" (UniqueName: \"kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc\") pod \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.639300 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.639260    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:15.694381 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.694323    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:15.946769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.946745    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"]
Sep 08 13:38:15.948675 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:15.948648    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a7a742_4f52_4362_9b17_861d9f8ff99b.slice/crio-ae55e2e0c230e5b9ab9ba83642fffca6509b8c2b8479d1147f4017c8d1f373cf WatchSource:0}: Error finding container ae55e2e0c230e5b9ab9ba83642fffca6509b8c2b8479d1147f4017c8d1f373cf: Status 404 returned error can't find the container with id ae55e2e0c230e5b9ab9ba83642fffca6509b8c2b8479d1147f4017c8d1f373cf
Sep 08 13:38:15.972802 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:15.972771    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw" event={"ID":"40a7a742-4f52-4362-9b17-861d9f8ff99b","Type":"ContainerStarted","Data":"ae55e2e0c230e5b9ab9ba83642fffca6509b8c2b8479d1147f4017c8d1f373cf"}
Sep 08 13:38:17.683541 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.683502    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zgwzk"]
Sep 08 13:38:17.687224 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.687196    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:17.690562 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.690523    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-lf7gz\""
Sep 08 13:38:17.702937 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.702917    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zgwzk"]
Sep 08 13:38:17.792952 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.792912    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c4k4p\" (UniqueName: \"kubernetes.io/projected/82b6390a-5a75-423a-bcf5-3724ffd8c799-kube-api-access-c4k4p\") pod \"authorino-operator-68d88948d8-zgwzk\" (UID: \"82b6390a-5a75-423a-bcf5-3724ffd8c799\") " pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:17.893702 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.893665    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c4k4p\" (UniqueName: \"kubernetes.io/projected/82b6390a-5a75-423a-bcf5-3724ffd8c799-kube-api-access-c4k4p\") pod \"authorino-operator-68d88948d8-zgwzk\" (UID: \"82b6390a-5a75-423a-bcf5-3724ffd8c799\") " pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:17.913855 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:17.913816    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c4k4p\" (UniqueName: \"kubernetes.io/projected/82b6390a-5a75-423a-bcf5-3724ffd8c799-kube-api-access-c4k4p\") pod \"authorino-operator-68d88948d8-zgwzk\" (UID: \"82b6390a-5a75-423a-bcf5-3724ffd8c799\") " pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:18.001111 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:18.001039    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:18.293181 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:18.293147    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-68d88948d8-zgwzk"]
Sep 08 13:38:18.295276 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:18.295248    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82b6390a_5a75_423a_bcf5_3724ffd8c799.slice/crio-ad79d098d6df52bd7f148d59a803c85af08d6cd94d025d869fc952daca8ae70b WatchSource:0}: Error finding container ad79d098d6df52bd7f148d59a803c85af08d6cd94d025d869fc952daca8ae70b: Status 404 returned error can't find the container with id ad79d098d6df52bd7f148d59a803c85af08d6cd94d025d869fc952daca8ae70b
Sep 08 13:38:18.994898 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:18.994854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk" event={"ID":"82b6390a-5a75-423a-bcf5-3724ffd8c799","Type":"ContainerStarted","Data":"ad79d098d6df52bd7f148d59a803c85af08d6cd94d025d869fc952daca8ae70b"}
Sep 08 13:38:19.026119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:19.026088    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:20.001958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:20.001883    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw" event={"ID":"40a7a742-4f52-4362-9b17-861d9f8ff99b","Type":"ContainerStarted","Data":"50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8"}
Sep 08 13:38:20.002456 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:20.002037    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:20.027441 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:20.027389    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw" podStartSLOduration=1.3499228730000001 podStartE2EDuration="5.027367804s" totalImagesPullingTime="3.677444931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:20.024239224 +0000 UTC m=+586.242155571" watchObservedRunningTime="2026-09-08 13:38:20.027367804 +0000 UTC m=+586.245284132"
Sep 08 13:38:20.110322 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:20.110285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:21.009329 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:21.009295    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk" event={"ID":"82b6390a-5a75-423a-bcf5-3724ffd8c799","Type":"ContainerStarted","Data":"de31a16002c5b0475dba80dc5f92640efd0dcaea87747d5d3c579bef71a57ce0"}
Sep 08 13:38:21.009776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:21.009366    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:21.034761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:21.034713    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk" podStartSLOduration=2.012760861 podStartE2EDuration="4.034699213s" totalImagesPullingTime="2.021938352s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:21.031435041 +0000 UTC m=+587.249351377" watchObservedRunningTime="2026-09-08 13:38:21.034699213 +0000 UTC m=+587.252615537"
Sep 08 13:38:21.808229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:21.808196    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:22.541134 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.541097    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"]
Sep 08 13:38:22.544956 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.544936    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.547255 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.547236    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 08 13:38:22.547653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.547639    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-vqss5\""
Sep 08 13:38:22.547797 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.547779    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 08 13:38:22.555774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.555751    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"]
Sep 08 13:38:22.639493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.639458    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63537a97-4519-468c-b170-c9c07f6a27eb-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.639741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.639538    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8fkn\" (UniqueName: \"kubernetes.io/projected/63537a97-4519-468c-b170-c9c07f6a27eb-kube-api-access-v8fkn\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.639741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.639594    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.719250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.719215    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:22.741023 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.740996    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8fkn\" (UniqueName: \"kubernetes.io/projected/63537a97-4519-468c-b170-c9c07f6a27eb-kube-api-access-v8fkn\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.741165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.741029    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.741165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.741122    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63537a97-4519-468c-b170-c9c07f6a27eb-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.741271 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:22.741236    2583 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Sep 08 13:38:22.741400 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:22.741361    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert podName:63537a97-4519-468c-b170-c9c07f6a27eb nodeName:}" failed. No retries permitted until 2026-09-08 13:38:23.241286568 +0000 UTC m=+589.459202884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert") pod "kuadrant-console-plugin-598f86468b-v44rd" (UID: "63537a97-4519-468c-b170-c9c07f6a27eb") : secret "plugin-serving-cert" not found
Sep 08 13:38:22.741828 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.741810    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/63537a97-4519-468c-b170-c9c07f6a27eb-nginx-conf\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.750685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.750663    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8fkn\" (UniqueName: \"kubernetes.io/projected/63537a97-4519-468c-b170-c9c07f6a27eb-kube-api-access-v8fkn\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:22.820044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.819960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:22.998273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:22.998239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:23.245990 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:23.245888    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:23.248352 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:23.248334    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/63537a97-4519-468c-b170-c9c07f6a27eb-plugin-serving-cert\") pod \"kuadrant-console-plugin-598f86468b-v44rd\" (UID: \"63537a97-4519-468c-b170-c9c07f6a27eb\") " pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:23.454836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:23.454804    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"
Sep 08 13:38:23.736071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:23.736041    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd"]
Sep 08 13:38:23.737719 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:23.737689    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod63537a97_4519_468c_b170_c9c07f6a27eb.slice/crio-8563c7b516e2612ad78bc3c669f7f80b318f2dead951044205a30a0c6ec2bf16 WatchSource:0}: Error finding container 8563c7b516e2612ad78bc3c669f7f80b318f2dead951044205a30a0c6ec2bf16: Status 404 returned error can't find the container with id 8563c7b516e2612ad78bc3c669f7f80b318f2dead951044205a30a0c6ec2bf16
Sep 08 13:38:23.963270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:23.963243    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:24.027683 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:24.027651    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd" event={"ID":"63537a97-4519-468c-b170-c9c07f6a27eb","Type":"ContainerStarted","Data":"8563c7b516e2612ad78bc3c669f7f80b318f2dead951044205a30a0c6ec2bf16"}
Sep 08 13:38:24.968624 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:24.968595    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-5985564776-lg74b"
Sep 08 13:38:26.579537 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:26.579498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:26.711812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:26.711756    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:26.797128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:26.797090    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:27.314144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:27.314100    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:30.068771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:30.068739    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd" event={"ID":"63537a97-4519-468c-b170-c9c07f6a27eb","Type":"ContainerStarted","Data":"8cb40e86877da6bb1e5fb7a3c261946fc8ec4717a30429e0ea1657f6c817c11e"}
Sep 08 13:38:31.011859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:31.011824    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:31.039539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:31.039493    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-598f86468b-v44rd" podStartSLOduration=3.57448904 podStartE2EDuration="9.039478031s" totalImagesPullingTime="5.464988991s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:30.096420068 +0000 UTC m=+596.314336396" watchObservedRunningTime="2026-09-08 13:38:31.039478031 +0000 UTC m=+597.257394356"
Sep 08 13:38:32.016728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.016698    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-68d88948d8-zgwzk"
Sep 08 13:38:32.693753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.693725    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"]
Sep 08 13:38:32.699009 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.698987    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.721653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.721627    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"]
Sep 08 13:38:32.738116 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.738085    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6t87\" (UniqueName: \"kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.738240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.738172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.838791 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.838755    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s6t87\" (UniqueName: \"kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.838968 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.838844    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.839184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.839167    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:32.856167 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:32.856147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6t87\" (UniqueName: \"kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87\") pod \"kuadrant-operator-controller-manager-68cf589589-gvwww\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:33.009389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.009298    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:33.293429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.293401    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"]
Sep 08 13:38:33.296820 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:33.296796    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3b0bfc3_de8b_4ad0_8b53_b80d3ac04d5e.slice/crio-2cf085e5c8160ab4624618bd587345f234ba9ed90c7e9681dd10a8c10f92d3e2 WatchSource:0}: Error finding container 2cf085e5c8160ab4624618bd587345f234ba9ed90c7e9681dd10a8c10f92d3e2: Status 404 returned error can't find the container with id 2cf085e5c8160ab4624618bd587345f234ba9ed90c7e9681dd10a8c10f92d3e2
Sep 08 13:38:33.324377 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.324346    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"]
Sep 08 13:38:33.324631 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.324586    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" containerName="manager" containerID="cri-o://50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8" gracePeriod=2
Sep 08 13:38:33.326796 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.326766    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.329292 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.329273    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"]
Sep 08 13:38:33.345734 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.345575    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"]
Sep 08 13:38:33.356301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.356275    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:33.356931 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.356910    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" containerName="manager"
Sep 08 13:38:33.356931 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.356931    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" containerName="manager"
Sep 08 13:38:33.357062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.357001    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" containerName="manager"
Sep 08 13:38:33.360289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.360272    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"]
Sep 08 13:38:33.360373 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.360359    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.368237 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.368215    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"]
Sep 08 13:38:33.368407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.368389    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" containerName="manager" containerID="cri-o://4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d" gracePeriod=2
Sep 08 13:38:33.373375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.373348    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:38:33.377857 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.377700    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.379827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.379808    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"]
Sep 08 13:38:33.382099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.382072    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:33.397249 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.397210    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:38:33.409430 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.409406    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"]
Sep 08 13:38:33.409820 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.409807    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" containerName="manager"
Sep 08 13:38:33.409866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.409822    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" containerName="manager"
Sep 08 13:38:33.409910 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.409893    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" containerName="manager"
Sep 08 13:38:33.412867 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.412849    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:33.422415 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.422386    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.424113 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.424091    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.425772 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.425750    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:33.437018 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.436952    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"]
Sep 08 13:38:33.444260 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.444239    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9kld\" (UniqueName: \"kubernetes.io/projected/ae98832b-209c-4ed7-a1ee-f44cb0a99e0a-kube-api-access-s9kld\") pod \"limitador-operator-controller-manager-6c549b6d85-5trd8\" (UID: \"ae98832b-209c-4ed7-a1ee-f44cb0a99e0a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:33.444375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.444302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjgn8\" (UniqueName: \"kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.444375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.444341    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lckrm\" (UniqueName: \"kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.444625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.444380    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.444625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.444447    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.545184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545117    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.545184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9kld\" (UniqueName: \"kubernetes.io/projected/ae98832b-209c-4ed7-a1ee-f44cb0a99e0a-kube-api-access-s9kld\") pod \"limitador-operator-controller-manager-6c549b6d85-5trd8\" (UID: \"ae98832b-209c-4ed7-a1ee-f44cb0a99e0a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:33.545398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545209    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjgn8\" (UniqueName: \"kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.545398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545242    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lckrm\" (UniqueName: \"kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.545398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545370    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.545653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.545821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.545803    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.557019 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.556996    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjgn8\" (UniqueName: \"kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8\") pod \"kuadrant-operator-controller-manager-68cf589589-p8mhl\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.557916 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.557897    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9kld\" (UniqueName: \"kubernetes.io/projected/ae98832b-209c-4ed7-a1ee-f44cb0a99e0a-kube-api-access-s9kld\") pod \"limitador-operator-controller-manager-6c549b6d85-5trd8\" (UID: \"ae98832b-209c-4ed7-a1ee-f44cb0a99e0a\") " pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:33.561506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.561483    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lckrm\" (UniqueName: \"kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm\") pod \"kuadrant-operator-controller-manager-58fc56757c-lqxdc\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.598624 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.598603    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.600686 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.600656    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:33.602360 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.602330    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.604834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.604816    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:33.607072 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.607053    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:33.608823 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:33.608804    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:33.645950 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.645928    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2l9kc\" (UniqueName: \"kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc\") pod \"40a7a742-4f52-4362-9b17-861d9f8ff99b\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") "
Sep 08 13:38:33.646118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.645996    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h9wpr\" (UniqueName: \"kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr\") pod \"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4\" (UID: \"296ece16-a4ea-40b0-acdc-c2c0dbf56ac4\") "
Sep 08 13:38:33.646118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.646054    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume\") pod \"40a7a742-4f52-4362-9b17-861d9f8ff99b\" (UID: \"40a7a742-4f52-4362-9b17-861d9f8ff99b\") "
Sep 08 13:38:33.646523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.646497    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "40a7a742-4f52-4362-9b17-861d9f8ff99b" (UID: "40a7a742-4f52-4362-9b17-861d9f8ff99b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:33.648219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.648199    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc" (OuterVolumeSpecName: "kube-api-access-2l9kc") pod "40a7a742-4f52-4362-9b17-861d9f8ff99b" (UID: "40a7a742-4f52-4362-9b17-861d9f8ff99b"). InnerVolumeSpecName "kube-api-access-2l9kc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:33.648219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.648209    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr" (OuterVolumeSpecName: "kube-api-access-h9wpr") pod "296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" (UID: "296ece16-a4ea-40b0-acdc-c2c0dbf56ac4"). InnerVolumeSpecName "kube-api-access-h9wpr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:33.746761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.746730    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2l9kc\" (UniqueName: \"kubernetes.io/projected/40a7a742-4f52-4362-9b17-861d9f8ff99b-kube-api-access-2l9kc\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:33.746761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.746756    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h9wpr\" (UniqueName: \"kubernetes.io/projected/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4-kube-api-access-h9wpr\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:33.746761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.746765    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40a7a742-4f52-4362-9b17-861d9f8ff99b-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:33.751589 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.751571    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:33.757660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.757644    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:33.766425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:33.766403    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:34.053141 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.053117    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:34.054610 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:34.054579    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaa4e35c_7ba5_4bed_bcdc_e204dccd50d5.slice/crio-90df4d11cfa373b40b61d6ead988d270ec465198c3e28a683700c5d4fc3708f5 WatchSource:0}: Error finding container 90df4d11cfa373b40b61d6ead988d270ec465198c3e28a683700c5d4fc3708f5: Status 404 returned error can't find the container with id 90df4d11cfa373b40b61d6ead988d270ec465198c3e28a683700c5d4fc3708f5
Sep 08 13:38:34.055691 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:34.055631    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda72aa3b8_9efa_4e58_af59_9ca539dbc5bb.slice/crio-7710f1bdd21ba536b2b07ecad1d0c67aad10d075401e0a3c521ce1e0ec28aeeb WatchSource:0}: Error finding container 7710f1bdd21ba536b2b07ecad1d0c67aad10d075401e0a3c521ce1e0ec28aeeb: Status 404 returned error can't find the container with id 7710f1bdd21ba536b2b07ecad1d0c67aad10d075401e0a3c521ce1e0ec28aeeb
Sep 08 13:38:34.056171 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.056145    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:38:34.066950 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.066930    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"]
Sep 08 13:38:34.068337 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:34.068303    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae98832b_209c_4ed7_a1ee_f44cb0a99e0a.slice/crio-362bee27abf75ce23d05f890a152f62a68bac74439b6bb4eb8c947f22a1a7bb6 WatchSource:0}: Error finding container 362bee27abf75ce23d05f890a152f62a68bac74439b6bb4eb8c947f22a1a7bb6: Status 404 returned error can't find the container with id 362bee27abf75ce23d05f890a152f62a68bac74439b6bb4eb8c947f22a1a7bb6
Sep 08 13:38:34.102343 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.102313    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" event={"ID":"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5","Type":"ContainerStarted","Data":"90df4d11cfa373b40b61d6ead988d270ec465198c3e28a683700c5d4fc3708f5"}
Sep 08 13:38:34.103752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.103726    2583 generic.go:356] "Generic (PLEG): container finished" podID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" containerID="4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d" exitCode=0
Sep 08 13:38:34.103873 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.103781    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.103943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.103880    2583 scope.go:117] "RemoveContainer" containerID="4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d"
Sep 08 13:38:34.105940 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.105856    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" event={"ID":"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb","Type":"ContainerStarted","Data":"7710f1bdd21ba536b2b07ecad1d0c67aad10d075401e0a3c521ce1e0ec28aeeb"}
Sep 08 13:38:34.106462 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.106431    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.108287 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.108240    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.110315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.110283    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8" event={"ID":"ae98832b-209c-4ed7-a1ee-f44cb0a99e0a","Type":"ContainerStarted","Data":"362bee27abf75ce23d05f890a152f62a68bac74439b6bb4eb8c947f22a1a7bb6"}
Sep 08 13:38:34.112846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.112825    2583 generic.go:356] "Generic (PLEG): container finished" podID="40a7a742-4f52-4362-9b17-861d9f8ff99b" containerID="50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8" exitCode=0
Sep 08 13:38:34.112934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.112920    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.115726 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.115670    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.115816 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.115711    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" containerName="manager" containerID="cri-o://53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3" gracePeriod=2
Sep 08 13:38:34.116681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.116663    2583 scope.go:117] "RemoveContainer" containerID="4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d"
Sep 08 13:38:34.116933 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.116911    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d\": container with ID starting with 4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d not found: ID does not exist" containerID="4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d"
Sep 08 13:38:34.117011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.116944    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d"} err="failed to get container status \"4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d\": rpc error: code = NotFound desc = could not find container \"4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d\": container with ID starting with 4695da2dfb22f560a51b92709b980a0fcc31748d7f49eb726ec1754e92ccb10d not found: ID does not exist"
Sep 08 13:38:34.117011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.116967    2583 scope.go:117] "RemoveContainer" containerID="50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8"
Sep 08 13:38:34.117728 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.117700    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.119914 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.119887    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-gvwww\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:34.121887 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.121866    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.123486 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.123466    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.165063 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.165030    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.166957 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.166928    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.167930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.167912    2583 scope.go:117] "RemoveContainer" containerID="50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8"
Sep 08 13:38:34.168214 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.168196    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8\": container with ID starting with 50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8 not found: ID does not exist" containerID="50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8"
Sep 08 13:38:34.168287 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.168221    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8"} err="failed to get container status \"50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8\": rpc error: code = NotFound desc = could not find container \"50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8\": container with ID starting with 50f1e3b7f1ddaf70017d683ed4341cc5d7256f1582430ded41443eef9169ecd8 not found: ID does not exist"
Sep 08 13:38:34.168998 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.168977    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-gvwww\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:34.193267 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.193247    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:38:34.193379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.193308    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:38:34.197053 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.197026    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.198764 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.198745    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.200360 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.200336    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-gvwww\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:34.232176 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.232146    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-6c549b6d85-xl55b\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-xl55b"
Sep 08 13:38:34.233863 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.233829    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-gvwww\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:34.236778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.236753    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="296ece16-a4ea-40b0-acdc-c2c0dbf56ac4" path="/var/lib/kubelet/pods/296ece16-a4ea-40b0-acdc-c2c0dbf56ac4/volumes"
Sep 08 13:38:34.237204 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.237184    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" path="/var/lib/kubelet/pods/40a7a742-4f52-4362-9b17-861d9f8ff99b/volumes"
Sep 08 13:38:34.237334 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:34.237307    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-58fc56757c-clzqw\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="40a7a742-4f52-4362-9b17-861d9f8ff99b" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-clzqw"
Sep 08 13:38:34.354088 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.354061    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:34.454354 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.454324    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume\") pod \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") "
Sep 08 13:38:34.454541 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.454417    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6t87\" (UniqueName: \"kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87\") pod \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\" (UID: \"a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e\") "
Sep 08 13:38:34.454683 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.454659    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" (UID: "a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:34.454802 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.454788    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:34.456518 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.456500    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87" (OuterVolumeSpecName: "kube-api-access-s6t87") pod "a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" (UID: "a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e"). InnerVolumeSpecName "kube-api-access-s6t87". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:34.555653 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:34.555627    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6t87\" (UniqueName: \"kubernetes.io/projected/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e-kube-api-access-s6t87\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:35.124355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.124256    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" event={"ID":"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb","Type":"ContainerStarted","Data":"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"}
Sep 08 13:38:35.124355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.124338    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:35.126064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.126033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8" event={"ID":"ae98832b-209c-4ed7-a1ee-f44cb0a99e0a","Type":"ContainerStarted","Data":"9cf15efde0e62903d8a5576002eb88e293f1b5a4c5f946522a0f477a693618b4"}
Sep 08 13:38:35.126208 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.126101    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:35.127403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.127379    2583 generic.go:356] "Generic (PLEG): container finished" podID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" containerID="53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3" exitCode=2
Sep 08 13:38:35.127513 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.127412    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:35.127513 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.127445    2583 scope.go:117] "RemoveContainer" containerID="53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3"
Sep 08 13:38:35.129386 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.129355    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" event={"ID":"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5","Type":"ContainerStarted","Data":"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"}
Sep 08 13:38:35.129486 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.129442    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:35.137060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.137046    2583 scope.go:117] "RemoveContainer" containerID="53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3"
Sep 08 13:38:35.137344 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:35.137326    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3\": container with ID starting with 53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3 not found: ID does not exist" containerID="53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3"
Sep 08 13:38:35.137387 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.137353    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3"} err="failed to get container status \"53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3\": rpc error: code = NotFound desc = could not find container \"53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3\": container with ID starting with 53fd9cd3a7a09dff1566d55d910cb53fecd41fb9d1c0218343a43617600a72c3 not found: ID does not exist"
Sep 08 13:38:35.160845 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.160804    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" podStartSLOduration=2.160791346 podStartE2EDuration="2.160791346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:35.158848518 +0000 UTC m=+601.376764862" watchObservedRunningTime="2026-09-08 13:38:35.160791346 +0000 UTC m=+601.378707671"
Sep 08 13:38:35.163391 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:35.163369    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-68cf589589-gvwww\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-gvwww"
Sep 08 13:38:35.211544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.211484    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" podStartSLOduration=2.211465275 podStartE2EDuration="2.211465275s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:35.208794138 +0000 UTC m=+601.426710488" watchObservedRunningTime="2026-09-08 13:38:35.211465275 +0000 UTC m=+601.429381603"
Sep 08 13:38:35.256160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.256110    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8" podStartSLOduration=2.2560910610000002 podStartE2EDuration="2.256091061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:35.254653366 +0000 UTC m=+601.472569683" watchObservedRunningTime="2026-09-08 13:38:35.256091061 +0000 UTC m=+601.474007386"
Sep 08 13:38:35.490064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:35.489977    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:36.232031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:36.231997    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" path="/var/lib/kubelet/pods/a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e/volumes"
Sep 08 13:38:36.370964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:36.370927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:38.041607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:38.041574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:38.128639 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:38.128596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:41.020886 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:41.020848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:42.407690 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:42.407658    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:45.931743 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:45.931716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:46.139197 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.139165    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:46.139379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.139341    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-6c549b6d85-5trd8"
Sep 08 13:38:46.139446 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.139404    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:46.294699 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.294663    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:46.294881 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.294855    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" podUID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" containerName="manager" containerID="cri-o://639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770" gracePeriod=10
Sep 08 13:38:46.432610 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.432581    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:46.557508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.557450    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:46.626314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.626282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:46.665484 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.665452    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume\") pod \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") "
Sep 08 13:38:46.665652 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.665502    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lckrm\" (UniqueName: \"kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm\") pod \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\" (UID: \"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb\") "
Sep 08 13:38:46.665849 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.665828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" (UID: "a72aa3b8-9efa-4e58-af59-9ca539dbc5bb"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:46.667745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.667725    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm" (OuterVolumeSpecName: "kube-api-access-lckrm") pod "a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" (UID: "a72aa3b8-9efa-4e58-af59-9ca539dbc5bb"). InnerVolumeSpecName "kube-api-access-lckrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:46.766911 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.766873    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lckrm\" (UniqueName: \"kubernetes.io/projected/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-kube-api-access-lckrm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:46.766911 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.766902    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:46.837879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.837794    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 13:38:46.838217 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838205    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" containerName="manager"
Sep 08 13:38:46.838272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838219    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" containerName="manager"
Sep 08 13:38:46.838272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838240    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" containerName="manager"
Sep 08 13:38:46.838272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838248    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" containerName="manager"
Sep 08 13:38:46.838391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838379    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3b0bfc3-de8b-4ad0-8b53-b80d3ac04d5e" containerName="manager"
Sep 08 13:38:46.838440 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.838399    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" containerName="manager"
Sep 08 13:38:46.841841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.841825    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:46.868467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.868440    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 13:38:46.968021 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.967979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:46.968474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.968246    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtpzm\" (UniqueName: \"kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:46.968474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:46.968387    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.069528 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.069491    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.069714 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.069582    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtpzm\" (UniqueName: \"kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.069925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.069905    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.099056 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.099000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtpzm\" (UniqueName: \"kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-vp2gv\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.151383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.151351    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:47.169252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.169199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:47.205480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.205300    2583 generic.go:356] "Generic (PLEG): container finished" podID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" containerID="639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770" exitCode=0
Sep 08 13:38:47.205480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.205353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" event={"ID":"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb","Type":"ContainerDied","Data":"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"}
Sep 08 13:38:47.205480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.205415    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"
Sep 08 13:38:47.205480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.205442    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc" event={"ID":"a72aa3b8-9efa-4e58-af59-9ca539dbc5bb","Type":"ContainerDied","Data":"7710f1bdd21ba536b2b07ecad1d0c67aad10d075401e0a3c521ce1e0ec28aeeb"}
Sep 08 13:38:47.205801 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.205495    2583 scope.go:117] "RemoveContainer" containerID="639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"
Sep 08 13:38:47.221089 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.221066    2583 scope.go:117] "RemoveContainer" containerID="639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"
Sep 08 13:38:47.221504 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:38:47.221484    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770\": container with ID starting with 639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770 not found: ID does not exist" containerID="639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"
Sep 08 13:38:47.221629 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.221513    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770"} err="failed to get container status \"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770\": rpc error: code = NotFound desc = could not find container \"639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770\": container with ID starting with 639152be77c3b96bf987cd9231393e6ce3a20f134f821bb803a21927b854c770 not found: ID does not exist"
Sep 08 13:38:47.267789 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.267758    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:47.284474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.284447    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-58fc56757c-lqxdc"]
Sep 08 13:38:47.475000 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:38:47.474968    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05aabb52_6e5d_4d50_bb5a_e04638f96dee.slice/crio-2b5d6d71ee70eccb462e9d95aeb427352d8d89ec6fbaefb13c7067cecc6f2e36 WatchSource:0}: Error finding container 2b5d6d71ee70eccb462e9d95aeb427352d8d89ec6fbaefb13c7067cecc6f2e36: Status 404 returned error can't find the container with id 2b5d6d71ee70eccb462e9d95aeb427352d8d89ec6fbaefb13c7067cecc6f2e36
Sep 08 13:38:47.475382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:47.475365    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 13:38:48.213228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:48.213179    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" event={"ID":"05aabb52-6e5d-4d50-bb5a-e04638f96dee","Type":"ContainerStarted","Data":"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"}
Sep 08 13:38:48.213718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:48.213242    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" event={"ID":"05aabb52-6e5d-4d50-bb5a-e04638f96dee","Type":"ContainerStarted","Data":"2b5d6d71ee70eccb462e9d95aeb427352d8d89ec6fbaefb13c7067cecc6f2e36"}
Sep 08 13:38:48.213718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:48.213278    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:48.230923 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:48.230892    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a72aa3b8-9efa-4e58-af59-9ca539dbc5bb" path="/var/lib/kubelet/pods/a72aa3b8-9efa-4e58-af59-9ca539dbc5bb/volumes"
Sep 08 13:38:48.263539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:48.263487    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" podStartSLOduration=2.263471026 podStartE2EDuration="2.263471026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:38:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:38:48.258170015 +0000 UTC m=+614.476086340" watchObservedRunningTime="2026-09-08 13:38:48.263471026 +0000 UTC m=+614.481387351"
Sep 08 13:38:54.801753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:54.801714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:54.907860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:54.907826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:38:59.222494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.222410    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 13:38:59.315310 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.315274    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:38:59.315506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.315482    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" podUID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" containerName="manager" containerID="cri-o://21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049" gracePeriod=10
Sep 08 13:38:59.571924 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.571903    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:38:59.679451 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.679415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume\") pod \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") "
Sep 08 13:38:59.679451 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.679455    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjgn8\" (UniqueName: \"kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8\") pod \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\" (UID: \"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5\") "
Sep 08 13:38:59.679918 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.679892    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" (UID: "aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:38:59.681829 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.681809    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8" (OuterVolumeSpecName: "kube-api-access-qjgn8") pod "aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" (UID: "aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5"). InnerVolumeSpecName "kube-api-access-qjgn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:38:59.780263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.780234    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:38:59.780263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:38:59.780262    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjgn8\" (UniqueName: \"kubernetes.io/projected/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5-kube-api-access-qjgn8\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:39:00.282989 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.282959    2583 generic.go:356] "Generic (PLEG): container finished" podID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" containerID="21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049" exitCode=0
Sep 08 13:39:00.283405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.283004    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"
Sep 08 13:39:00.283405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.283051    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" event={"ID":"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5","Type":"ContainerDied","Data":"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"}
Sep 08 13:39:00.283405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.283101    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl" event={"ID":"aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5","Type":"ContainerDied","Data":"90df4d11cfa373b40b61d6ead988d270ec465198c3e28a683700c5d4fc3708f5"}
Sep 08 13:39:00.283405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.283124    2583 scope.go:117] "RemoveContainer" containerID="21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"
Sep 08 13:39:00.296119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.296101    2583 scope.go:117] "RemoveContainer" containerID="21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"
Sep 08 13:39:00.296401 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:39:00.296381    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049\": container with ID starting with 21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049 not found: ID does not exist" containerID="21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"
Sep 08 13:39:00.296474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.296411    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049"} err="failed to get container status \"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049\": rpc error: code = NotFound desc = could not find container \"21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049\": container with ID starting with 21122cecc4cc80675fff28d9821cb1c50ced792a048c39cfc94fea8a8bc6e049 not found: ID does not exist"
Sep 08 13:39:00.308998 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.308971    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:39:00.316207 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:00.316185    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-68cf589589-p8mhl"]
Sep 08 13:39:02.231179 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:02.231144    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" path="/var/lib/kubelet/pods/aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5/volumes"
Sep 08 13:39:03.002806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:03.002771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:03.417726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:03.417695    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:16.792062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:16.792021    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:16.892702 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:16.892665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:18.028896 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.028863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:18.051443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.051414    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:18.051883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.051868    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" containerName="manager"
Sep 08 13:39:18.051930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.051884    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" containerName="manager"
Sep 08 13:39:18.051975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.051950    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa4e35c-7ba5-4bed-bcdc-e204dccd50d5" containerName="manager"
Sep 08 13:39:18.073498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.073463    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:18.073660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.073570    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.076996 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.076978    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 08 13:39:18.137943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.137912    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:18.146203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.146173    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.146373 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.146291    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z2vh8\" (UniqueName: \"kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.151374 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.151347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:18.246903 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.246872    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.247104 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.246953    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z2vh8\" (UniqueName: \"kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.247674 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.247650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.250705 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.250649    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:18.255434 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.255411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z2vh8\" (UniqueName: \"kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8\") pod \"limitador-limitador-859c4d96d6-7djhp\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") " pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.384374 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.384283    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:18.485059 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.485018    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:39:18.488114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.488095    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.507919 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.507879    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:39:18.510280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.510247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:18.550067 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.550039    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a240ca1-8435-4e0d-871b-d0cec30e1527-config-file\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.550229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.550078    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k54jj\" (UniqueName: \"kubernetes.io/projected/2a240ca1-8435-4e0d-871b-d0cec30e1527-kube-api-access-k54jj\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.562948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.562921    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:39:18.651447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.651369    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a240ca1-8435-4e0d-871b-d0cec30e1527-config-file\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.651447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.651423    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k54jj\" (UniqueName: \"kubernetes.io/projected/2a240ca1-8435-4e0d-871b-d0cec30e1527-kube-api-access-k54jj\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.652043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.652026    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/2a240ca1-8435-4e0d-871b-d0cec30e1527-config-file\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.660817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.660783    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k54jj\" (UniqueName: \"kubernetes.io/projected/2a240ca1-8435-4e0d-871b-d0cec30e1527-kube-api-access-k54jj\") pod \"limitador-limitador-85d9db4d5d-8nbgf\" (UID: \"2a240ca1-8435-4e0d-871b-d0cec30e1527\") " pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.671675 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.671654    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:18.673759 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:39:18.673735    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36cb116b_4ac4_4c06_baa1_38d9eb7773d3.slice/crio-d88d20796648f8c2b0ed539411718167548848589119cf471661211f2be4b62a WatchSource:0}: Error finding container d88d20796648f8c2b0ed539411718167548848589119cf471661211f2be4b62a: Status 404 returned error can't find the container with id d88d20796648f8c2b0ed539411718167548848589119cf471661211f2be4b62a
Sep 08 13:39:18.799372 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.799330    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:18.989247 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:18.989159    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:19.080420 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:19.080395    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:39:19.083021 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:39:19.082994    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a240ca1_8435_4e0d_871b_d0cec30e1527.slice/crio-ae9d57e7d079337ff230c97087f116388429518edc7448c450e1e1e262e3fc7e WatchSource:0}: Error finding container ae9d57e7d079337ff230c97087f116388429518edc7448c450e1e1e262e3fc7e: Status 404 returned error can't find the container with id ae9d57e7d079337ff230c97087f116388429518edc7448c450e1e1e262e3fc7e
Sep 08 13:39:19.100141 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:19.100112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:19.405930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:19.405887    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf" event={"ID":"2a240ca1-8435-4e0d-871b-d0cec30e1527","Type":"ContainerStarted","Data":"ae9d57e7d079337ff230c97087f116388429518edc7448c450e1e1e262e3fc7e"}
Sep 08 13:39:19.407237 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:19.407213    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" event={"ID":"36cb116b-4ac4-4c06-baa1-38d9eb7773d3","Type":"ContainerStarted","Data":"d88d20796648f8c2b0ed539411718167548848589119cf471661211f2be4b62a"}
Sep 08 13:39:20.414834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.414802    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf" event={"ID":"2a240ca1-8435-4e0d-871b-d0cec30e1527","Type":"ContainerStarted","Data":"55003a438e9870b17cf02d939f084c8a4236cd249b46b6af485aaa50045b8d03"}
Sep 08 13:39:20.415245 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.414903    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:20.416568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.416535    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" event={"ID":"36cb116b-4ac4-4c06-baa1-38d9eb7773d3","Type":"ContainerStarted","Data":"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"}
Sep 08 13:39:20.416667 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.416598    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:20.434745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.434692    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf" podStartSLOduration=1.618292126 podStartE2EDuration="2.434678659s" totalImagesPullingTime="816.386533ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:39:20.432595911 +0000 UTC m=+646.650512260" watchObservedRunningTime="2026-09-08 13:39:20.434678659 +0000 UTC m=+646.652594977"
Sep 08 13:39:20.452726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:20.452687    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" podStartSLOduration=1.270071119 podStartE2EDuration="2.452673507s" totalImagesPullingTime="1.182602388s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:39:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:39:20.451268638 +0000 UTC m=+646.669184974" watchObservedRunningTime="2026-09-08 13:39:20.452673507 +0000 UTC m=+646.670589853"
Sep 08 13:39:31.423274 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:31.423245    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:31.423775 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:31.423310    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"
Sep 08 13:39:31.500293 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:31.500257    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:31.500498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:31.500472    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" podUID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" containerName="limitador" containerID="cri-o://a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5" gracePeriod=30
Sep 08 13:39:32.450781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.450758    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:32.486520 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.486484    2583 generic.go:356] "Generic (PLEG): container finished" podID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" containerID="a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5" exitCode=0
Sep 08 13:39:32.486733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.486528    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp"
Sep 08 13:39:32.486733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.486532    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" event={"ID":"36cb116b-4ac4-4c06-baa1-38d9eb7773d3","Type":"ContainerDied","Data":"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"}
Sep 08 13:39:32.486733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.486613    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-859c4d96d6-7djhp" event={"ID":"36cb116b-4ac4-4c06-baa1-38d9eb7773d3","Type":"ContainerDied","Data":"d88d20796648f8c2b0ed539411718167548848589119cf471661211f2be4b62a"}
Sep 08 13:39:32.486733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.486644    2583 scope.go:117] "RemoveContainer" containerID="a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"
Sep 08 13:39:32.497844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.497825    2583 scope.go:117] "RemoveContainer" containerID="a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"
Sep 08 13:39:32.498159 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:39:32.498139    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5\": container with ID starting with a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5 not found: ID does not exist" containerID="a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"
Sep 08 13:39:32.498230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.498171    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5"} err="failed to get container status \"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5\": rpc error: code = NotFound desc = could not find container \"a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5\": container with ID starting with a3bf5ef80150b0cd8ed6c048537ad40b664e11e6c126d265cb1069bbb477bcc5 not found: ID does not exist"
Sep 08 13:39:32.573702 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.573604    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file\") pod \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") "
Sep 08 13:39:32.573876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.573774    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z2vh8\" (UniqueName: \"kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8\") pod \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\" (UID: \"36cb116b-4ac4-4c06-baa1-38d9eb7773d3\") "
Sep 08 13:39:32.573966 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.573940    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file" (OuterVolumeSpecName: "config-file") pod "36cb116b-4ac4-4c06-baa1-38d9eb7773d3" (UID: "36cb116b-4ac4-4c06-baa1-38d9eb7773d3"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:39:32.574060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.574044    2583 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-config-file\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:39:32.575951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.575926    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8" (OuterVolumeSpecName: "kube-api-access-z2vh8") pod "36cb116b-4ac4-4c06-baa1-38d9eb7773d3" (UID: "36cb116b-4ac4-4c06-baa1-38d9eb7773d3"). InnerVolumeSpecName "kube-api-access-z2vh8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:39:32.675060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.675018    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z2vh8\" (UniqueName: \"kubernetes.io/projected/36cb116b-4ac4-4c06-baa1-38d9eb7773d3-kube-api-access-z2vh8\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:39:32.810472 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.810439    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:32.817266 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:32.817239    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-859c4d96d6-7djhp"]
Sep 08 13:39:34.231733 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:34.231698    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" path="/var/lib/kubelet/pods/36cb116b-4ac4-4c06-baa1-38d9eb7773d3/volumes"
Sep 08 13:39:36.502355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:36.502324    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:36.644402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:36.644366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:37.054686 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:37.054656    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:49.363466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:49.363433    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:49.950166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:49.950136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:55.350954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:55.350912    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:55.451329 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:55.451295    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:56.754338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:56.754309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:56.851324 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:56.851286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:56.946422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:56.946387    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:57.028523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:57.028493    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:39:57.100886 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:39:57.100854    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:00.371064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:00.371028    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:00.479402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:00.479370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:32.029138 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:32.029043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:32.236678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:32.236647    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:33.608370 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.608333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:33.709531 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.709492    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:33.796998 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.796959    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 13:40:33.797608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.797546    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" containerName="limitador"
Sep 08 13:40:33.797608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.797585    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" containerName="limitador"
Sep 08 13:40:33.797608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.797696    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="36cb116b-4ac4-4c06-baa1-38d9eb7773d3" containerName="limitador"
Sep 08 13:40:33.801156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.801135    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:33.804543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.804521    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzrks\""
Sep 08 13:40:33.804666 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.804613    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 08 13:40:33.804666 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.804623    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 08 13:40:33.804742 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.804521    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 08 13:40:33.804779 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.804756    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 08 13:40:33.819773 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.819750    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 13:40:33.839882 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.839853    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:33.929120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.929028    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g44mr\" (UniqueName: \"kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:33.929120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.929071    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:33.929316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.929165    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:33.929316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.929225    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:33.941388 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:33.941359    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:34.030147 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.030114    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g44mr\" (UniqueName: \"kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.030308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.030164    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.030308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.030204    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.030308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.030249    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.030850 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.030813    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.032997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.032965    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.032997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.032979    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.040892 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.040872    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g44mr\" (UniqueName: \"kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr\") pod \"maas-api-84d9df9944-2tvc5\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") " pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.116417 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.116385    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-fzrks\""
Sep 08 13:40:34.123162 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.123122    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:34.275622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.275590    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:34.389906 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.389871    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:34.447747 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.447723    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 13:40:34.450016 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:40:34.449984    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc28ae82_9021_48d0_93db_0522b4f15253.slice/crio-a4bb210fc357b7acdd7bbf337d208a103d64fda8614a0e30866023505c7374f2 WatchSource:0}: Error finding container a4bb210fc357b7acdd7bbf337d208a103d64fda8614a0e30866023505c7374f2: Status 404 returned error can't find the container with id a4bb210fc357b7acdd7bbf337d208a103d64fda8614a0e30866023505c7374f2
Sep 08 13:40:34.451339 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.451323    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:40:34.533405 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.533366    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:40:34.852892 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:34.852854    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" event={"ID":"fc28ae82-9021-48d0-93db-0522b4f15253","Type":"ContainerStarted","Data":"a4bb210fc357b7acdd7bbf337d208a103d64fda8614a0e30866023505c7374f2"}
Sep 08 13:40:37.871298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:37.871257    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" event={"ID":"fc28ae82-9021-48d0-93db-0522b4f15253","Type":"ContainerStarted","Data":"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"}
Sep 08 13:40:37.871865 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:37.871317    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:37.891375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:37.891323    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" podStartSLOduration=2.535890699 podStartE2EDuration="4.891309184s" totalImagesPullingTime="2.355418485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:40:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:40:37.890515304 +0000 UTC m=+724.108431657" watchObservedRunningTime="2026-09-08 13:40:37.891309184 +0000 UTC m=+724.109225546"
Sep 08 13:40:43.882379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:43.882349    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 13:40:44.905271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:44.905229    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:45.002680 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:45.002644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:59.715008 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:59.714954    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:40:59.789467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:40:59.789425    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:03.095155 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:03.095112    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:08.330934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.330899    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"]
Sep 08 13:41:08.335227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.335207    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.337833 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.337803    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 08 13:41:08.338643 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.338619    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 08 13:41:08.338730 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.338645    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ws2qv\""
Sep 08 13:41:08.338730 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.338653    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 08 13:41:08.345304 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.345282    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"]
Sep 08 13:41:08.377261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.377219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:08.457818 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.457774    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.457985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.457824    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.457985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.457941    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.457985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.457977    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbh7c\" (UniqueName: \"kubernetes.io/projected/d70e92b1-286b-423a-9967-afea26e104da-kube-api-access-gbh7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.458127 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.457999    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.458127 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.458017    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d70e92b1-286b-423a-9967-afea26e104da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559363    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559413    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbh7c\" (UniqueName: \"kubernetes.io/projected/d70e92b1-286b-423a-9967-afea26e104da-kube-api-access-gbh7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559535    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559594    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d70e92b1-286b-423a-9967-afea26e104da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559810 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559655    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559810 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559678    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559846    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559863    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.559965 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.559951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.562092 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.562068    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d70e92b1-286b-423a-9967-afea26e104da-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.562424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.562406    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d70e92b1-286b-423a-9967-afea26e104da-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.566979 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.566958    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbh7c\" (UniqueName: \"kubernetes.io/projected/d70e92b1-286b-423a-9967-afea26e104da-kube-api-access-gbh7c\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp\" (UID: \"d70e92b1-286b-423a-9967-afea26e104da\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.648099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.648011    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:08.849219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.849178    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:41:08.926722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:08.926697    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"]
Sep 08 13:41:08.928709 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:41:08.928684    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd70e92b1_286b_423a_9967_afea26e104da.slice/crio-d55e39d1b80222953a43ee876ab925e40378aab2ab1ec2f038380dc32a4a8d21 WatchSource:0}: Error finding container d55e39d1b80222953a43ee876ab925e40378aab2ab1ec2f038380dc32a4a8d21: Status 404 returned error can't find the container with id d55e39d1b80222953a43ee876ab925e40378aab2ab1ec2f038380dc32a4a8d21
Sep 08 13:41:09.050967 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:09.050932    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp" event={"ID":"d70e92b1-286b-423a-9967-afea26e104da","Type":"ContainerStarted","Data":"d55e39d1b80222953a43ee876ab925e40378aab2ab1ec2f038380dc32a4a8d21"}
Sep 08 13:41:10.060173 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:10.060141    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp" event={"ID":"d70e92b1-286b-423a-9967-afea26e104da","Type":"ContainerStarted","Data":"aa2bc2a24900fe54ff93a489ebfb2b186dc849e2e2ade887d698bf73a244f965"}
Sep 08 13:41:10.079730 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:10.079670    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp" podStartSLOduration=1.072660147 podStartE2EDuration="2.079651755s" totalImagesPullingTime="1.006991608s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:41:10.077363308 +0000 UTC m=+756.295279657" watchObservedRunningTime="2026-09-08 13:41:10.079651755 +0000 UTC m=+756.297568080"
Sep 08 13:41:18.649158 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:18.649119    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:18.649601 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:18.649219    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:18.661605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:18.661583    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:19.124080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:19.124052    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp"
Sep 08 13:41:23.685417 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:23.685372    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:24.352819 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:24.352789    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:41:45.754734 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:45.754696    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:56.704487 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:56.704393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:41:57.700944 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:41:57.700906    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:42:02.274879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:02.274848    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:42:03.544158 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:03.544118    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:42:05.637883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.637852    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"]
Sep 08 13:42:05.641265 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.641249    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.643429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.643414    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Sep 08 13:42:05.653932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.653910    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"]
Sep 08 13:42:05.675163 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.675130    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:42:05.703508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703480    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.703720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703525    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wgnrl\" (UniqueName: \"kubernetes.io/projected/8ba31e5c-f07b-4d6a-ba04-765599932f6f-kube-api-access-wgnrl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.703720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703621    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.703720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703676    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.703859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703736    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.703859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.703790    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804236 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804195    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804246    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804277    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wgnrl\" (UniqueName: \"kubernetes.io/projected/8ba31e5c-f07b-4d6a-ba04-765599932f6f-kube-api-access-wgnrl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804337    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804772 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804442    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804772 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804711    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804849 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804776    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.804905 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.804884    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.806780 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.806760    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8ba31e5c-f07b-4d6a-ba04-765599932f6f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.807041 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.807023    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8ba31e5c-f07b-4d6a-ba04-765599932f6f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.813223 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.813199    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wgnrl\" (UniqueName: \"kubernetes.io/projected/8ba31e5c-f07b-4d6a-ba04-765599932f6f-kube-api-access-wgnrl\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb\" (UID: \"8ba31e5c-f07b-4d6a-ba04-765599932f6f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:05.953089 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:05.953003    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:06.232079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:06.232050    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"]
Sep 08 13:42:06.232621 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:42:06.232595    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ba31e5c_f07b_4d6a_ba04_765599932f6f.slice/crio-0d9434e8e9e8b744302fd3dd5d4f8547d1512b6dbaabb9bf924da6f765c43a93 WatchSource:0}: Error finding container 0d9434e8e9e8b744302fd3dd5d4f8547d1512b6dbaabb9bf924da6f765c43a93: Status 404 returned error can't find the container with id 0d9434e8e9e8b744302fd3dd5d4f8547d1512b6dbaabb9bf924da6f765c43a93
Sep 08 13:42:06.391177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:06.391136    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb" event={"ID":"8ba31e5c-f07b-4d6a-ba04-765599932f6f","Type":"ContainerStarted","Data":"0d9434e8e9e8b744302fd3dd5d4f8547d1512b6dbaabb9bf924da6f765c43a93"}
Sep 08 13:42:07.398892 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:07.398850    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb" event={"ID":"8ba31e5c-f07b-4d6a-ba04-765599932f6f","Type":"ContainerStarted","Data":"7e38e68371ece9e46d77e2db7a13333268789c5ba758f4e0615308394f92ec1d"}
Sep 08 13:42:07.420119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:07.420056    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb" podStartSLOduration=2.208186294 podStartE2EDuration="2.420039171s" totalImagesPullingTime="211.852877ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:42:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:42:07.418404151 +0000 UTC m=+813.636320479" watchObservedRunningTime="2026-09-08 13:42:07.420039171 +0000 UTC m=+813.637955554"
Sep 08 13:42:07.549851 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:07.549815    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:42:14.466043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:14.466003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:42:15.953353 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:15.953309    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:15.953877 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:15.953381    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:15.966257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:15.966228    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:16.146677 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:16.146642    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:42:16.464339 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:16.464304    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb"
Sep 08 13:42:42.947749 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:42.947711    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:42:44.951900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:42:44.951862    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:43:12.453796 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:12.453712    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:34.241617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:34.241590    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:43:34.244469 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:34.244451    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:43:38.236972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:38.236936    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:38.333333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:38.333298    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:39.417744 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:39.417715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:39.554926 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:39.554890    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:56.562443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.562400    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:56.914582 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.914490    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:43:56.918024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.918005    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:56.922110 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.922086    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 08 13:43:56.922262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.922086    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-2d1742-openshift-default-dockercfg-6ddwv\""
Sep 08 13:43:56.932686 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:56.932659    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:43:57.071741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071701    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.071741 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071744    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071839    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071891    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071928    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sksxd\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071944    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.071963    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.072329 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.072008    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.172983 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.172886    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.172983 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.172958    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.172983 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.172980    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.172999    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173018    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173066    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173120    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sksxd\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173146    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173183    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173342    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173650    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173760 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173698    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173811    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.173896 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.173869    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.175823 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.175799    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.175978 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.175959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.181305 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.181271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.181464 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.181411    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sksxd\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd\") pod \"e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") " pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.230290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.230255    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:43:57.518168 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.518144    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:43:57.520612 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:43:57.520575    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a2f44b2_fd79_4cfb_b592_8a31268c716b.slice/crio-d72996657c839d72f23d205279711d963f6fd9d3f5cf9437d304e59cdcbf5295 WatchSource:0}: Error finding container d72996657c839d72f23d205279711d963f6fd9d3f5cf9437d304e59cdcbf5295: Status 404 returned error can't find the container with id d72996657c839d72f23d205279711d963f6fd9d3f5cf9437d304e59cdcbf5295
Sep 08 13:43:57.996048 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:57.996002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:58.068119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:58.068078    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerStarted","Data":"d72996657c839d72f23d205279711d963f6fd9d3f5cf9437d304e59cdcbf5295"}
Sep 08 13:43:58.147342 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:58.147307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:58.249093 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:58.248996    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:43:58.766850 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:43:58.766812    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:00.498993 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:00.498957    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:00.499273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:00.499030    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:00.499273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:00.499057    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:00.748159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:00.748077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:44:01.093608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:01.093567    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerStarted","Data":"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"}
Sep 08 13:44:01.119591 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:01.119502    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podStartSLOduration=2.143626376 podStartE2EDuration="5.119480322s" totalImagesPullingTime="2.975853946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:01.116740666 +0000 UTC m=+927.334657017" watchObservedRunningTime="2026-09-08 13:44:01.119480322 +0000 UTC m=+927.337396649"
Sep 08 13:44:01.230568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:01.230522    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:01.235831 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:01.235805    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:02.100067 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:02.100033    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:02.101201 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:02.101177    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:03.340024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:03.339984    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:03.486232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:03.486194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:04.595061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.595028    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:44:04.599362 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.599339    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.603736 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.603709    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-2d1742-serving-cert\""
Sep 08 13:44:04.616013 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.615980    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:44:04.642636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.642604    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:04.734168 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.734120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:04.753421 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.753383    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.753628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.753499    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.753628 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.753571    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.753759 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.753645    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chwb8\" (UniqueName: \"kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.854276 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.854181    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.854276 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.854268    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.854526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.854322    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.854526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.854358    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chwb8\" (UniqueName: \"kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.855164 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.855084    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.856976 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.856946    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.856976 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.856959    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.866307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.866271    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chwb8\" (UniqueName: \"kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8\") pod \"maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:04.893467 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.893438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:04.910448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:04.910420    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:05.171515 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:05.171437    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:05.200686 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:05.200566    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:44:05.202942 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:44:05.202912    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cc0ee64_c6bd_44c9_a2dd_91231f9c4bb3.slice/crio-08722ae8240fe221c0ee11985ad723abc6f9bee4f09bb746bf2d5557bc6150c3 WatchSource:0}: Error finding container 08722ae8240fe221c0ee11985ad723abc6f9bee4f09bb746bf2d5557bc6150c3: Status 404 returned error can't find the container with id 08722ae8240fe221c0ee11985ad723abc6f9bee4f09bb746bf2d5557bc6150c3
Sep 08 13:44:05.269781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:05.269742    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:05.949732 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:05.949641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:06.049986 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.049950    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:06.126068 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.126033    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" event={"ID":"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3","Type":"ContainerStarted","Data":"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"}
Sep 08 13:44:06.126288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.126093    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" event={"ID":"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3","Type":"ContainerStarted","Data":"08722ae8240fe221c0ee11985ad723abc6f9bee4f09bb746bf2d5557bc6150c3"}
Sep 08 13:44:06.126288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.126209    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:06.150459 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.150404    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" podStartSLOduration=1.664649955 podStartE2EDuration="2.150386731s" totalImagesPullingTime="485.736776ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:06.14926107 +0000 UTC m=+932.367177413" watchObservedRunningTime="2026-09-08 13:44:06.150386731 +0000 UTC m=+932.368303056"
Sep 08 13:44:06.184421 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.184384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:06.556831 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.556797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:06.639332 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:06.639302    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:07.515178 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:07.515143    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:07.898621 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:07.898586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:08.045751 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:08.045718    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:11.185840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:11.185803    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:11.278298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:11.278262    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:11.848809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:11.848768    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:12.138849 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.138775    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:44:12.546543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.546506    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:12.550925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.550897    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.553888 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.553861    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0b415db2-serving-cert\""
Sep 08 13:44:12.562967 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.562934    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:12.618738 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.618702    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:12.624822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.624787    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-476g4\" (UniqueName: \"kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.624988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.624830    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.624988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.624902    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.625087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.624990    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.723595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.723539    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:12.725666 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.725643    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-476g4\" (UniqueName: \"kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.725753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.725674    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.725828 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.725811    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.725931 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.725914    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.726458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.726440    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.728082 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.728058    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.728246 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.728226    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.738154 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.738128    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-476g4\" (UniqueName: \"kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4\") pod \"maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:12.867272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:12.867173    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:13.168756 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.168724    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:13.169877 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:44:13.169839    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28991251_daef_4679_9d9a_258f27800d34.slice/crio-b9325c3803320fb0362dbc37b8ba9c2f7a4566db2cf7ed6a593b30b6ff1a2dd2 WatchSource:0}: Error finding container b9325c3803320fb0362dbc37b8ba9c2f7a4566db2cf7ed6a593b30b6ff1a2dd2: Status 404 returned error can't find the container with id b9325c3803320fb0362dbc37b8ba9c2f7a4566db2cf7ed6a593b30b6ff1a2dd2
Sep 08 13:44:13.210328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.209205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:13.313846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.313810    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:13.641748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.641711    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:44:13.647505 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.647481    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.650314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.650286    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-b719d2-serving-cert\""
Sep 08 13:44:13.663608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.663578    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:44:13.707526 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.707489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:13.735597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.735562    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.735787 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.735610    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqnmw\" (UniqueName: \"kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.735787 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.735652    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.735787 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.735672    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.789189 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.788997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:13.836964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.836929    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.837170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.836975    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqnmw\" (UniqueName: \"kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.837170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.837015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.837170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.837115    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.837392 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:13.837172    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-serving-cert: secret "maas-api-e2e-autores-b719d2-serving-cert" not found
Sep 08 13:44:13.837392 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:13.837242    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls podName:8b4f0440-2f82-4470-a2be-c44f62aa5412 nodeName:}" failed. No retries permitted until 2026-09-08 13:44:14.3372202 +0000 UTC m=+940.555136567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls") pod "maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412") : secret "maas-api-e2e-autores-b719d2-serving-cert" not found
Sep 08 13:44:13.837898 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.837874    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.840176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.840153    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:13.850678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:13.850647    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqnmw\" (UniqueName: \"kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:14.183597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.183413    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" event={"ID":"28991251-daef-4679-9d9a-258f27800d34","Type":"ContainerStarted","Data":"b9325c3803320fb0362dbc37b8ba9c2f7a4566db2cf7ed6a593b30b6ff1a2dd2"}
Sep 08 13:44:14.203356 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.203272    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:14.342404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.342311    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:14.342596 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:14.342495    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-serving-cert: secret "maas-api-e2e-autores-b719d2-serving-cert" not found
Sep 08 13:44:14.342665 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:14.342595    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls podName:8b4f0440-2f82-4470-a2be-c44f62aa5412 nodeName:}" failed. No retries permitted until 2026-09-08 13:44:15.342572367 +0000 UTC m=+941.560488678 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls") pod "maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412") : secret "maas-api-e2e-autores-b719d2-serving-cert" not found
Sep 08 13:44:14.348876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.348811    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:14.435568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.435506    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:14.613069 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:14.612976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:15.191781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.191735    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" event={"ID":"28991251-daef-4679-9d9a-258f27800d34","Type":"ContainerStarted","Data":"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"}
Sep 08 13:44:15.192263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.191826    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:15.210569 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.210509    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" podStartSLOduration=2.184301319 podStartE2EDuration="3.21049094s" totalImagesPullingTime="1.026189621s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:15.20780837 +0000 UTC m=+941.425724747" watchObservedRunningTime="2026-09-08 13:44:15.21049094 +0000 UTC m=+941.428407265"
Sep 08 13:44:15.353259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.353218    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:15.356023 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.355994    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"maas-api-e2e-autores-b719d2-69f9b65457-lmwdh\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:15.406805 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.406762    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:15.459640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.459541    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:15.743986 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:15.743959    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:44:15.745894 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:44:15.745865    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4f0440_2f82_4470_a2be_c44f62aa5412.slice/crio-198460756e8f20b332eed194b8aa0e90f4d211260215087e30f0c7ecacbe6c80 WatchSource:0}: Error finding container 198460756e8f20b332eed194b8aa0e90f4d211260215087e30f0c7ecacbe6c80: Status 404 returned error can't find the container with id 198460756e8f20b332eed194b8aa0e90f4d211260215087e30f0c7ecacbe6c80
Sep 08 13:44:16.199654 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:16.199600    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" event={"ID":"8b4f0440-2f82-4470-a2be-c44f62aa5412","Type":"ContainerStarted","Data":"198460756e8f20b332eed194b8aa0e90f4d211260215087e30f0c7ecacbe6c80"}
Sep 08 13:44:16.347105 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:16.347069    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:44:16.356061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:16.356026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:16.425310 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:16.425219    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:17.209482 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:17.209443    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" event={"ID":"8b4f0440-2f82-4470-a2be-c44f62aa5412","Type":"ContainerStarted","Data":"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"}
Sep 08 13:44:17.209990 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:17.209511    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:17.229936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:17.229879    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" podStartSLOduration=3.825899571 podStartE2EDuration="4.229862225s" totalImagesPullingTime="403.962654ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:17.22781209 +0000 UTC m=+943.445728429" watchObservedRunningTime="2026-09-08 13:44:17.229862225 +0000 UTC m=+943.447778549"
Sep 08 13:44:17.393148 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:17.392425    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:34466->10.133.0.59:15021: read: connection reset by peer" start-of-body=
Sep 08 13:44:17.393148 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:17.392498    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": read tcp 10.133.0.2:34466->10.133.0.59:15021: read: connection reset by peer"
Sep 08 13:44:18.217666 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.217626    2583 generic.go:356] "Generic (PLEG): container finished" podID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerID="5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943" exitCode=0
Sep 08 13:44:18.218040 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.217705    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerDied","Data":"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"}
Sep 08 13:44:18.218231 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.218214    2583 scope.go:117] "RemoveContainer" containerID="5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"
Sep 08 13:44:18.219253 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.219226    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:18.219306 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.219293    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:18.219346 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.219330    2583 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892144Ki","pods":"250"}
Sep 08 13:44:18.913771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:18.913740    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:19.226812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:19.226723    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerStarted","Data":"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"}
Sep 08 13:44:19.230543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:19.230523    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:19.230737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:19.230718    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:19.230798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:19.230759    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:20.231375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:20.231334    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:20.231815 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:20.231400    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:20.674698 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:20.674667    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:21.206052 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:21.206016    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:21.230639 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:21.230602    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:21.230846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:21.230665    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:22.012879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:22.012838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:22.107809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:22.107771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:22.231307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:22.231273    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:22.231497 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:22.231366    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:22.275803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:22.275763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:23.224054 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:23.223973    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:44:23.231005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:23.230981    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:23.231139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:23.231033    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:24.231668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:24.231636    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused" start-of-body=
Sep 08 13:44:24.232070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:24.231693    2583 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.59:15021/healthz/ready\": dial tcp 10.133.0.59:15021: connect: connection refused"
Sep 08 13:44:25.235020 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:25.234986    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:25.235415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:25.235321    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:25.236177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:25.236160    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:44:25.777263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:25.777205    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:28.350256 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:28.350218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:28.768372 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:28.768291    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:28.999970 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:28.999930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:29.115392 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:29.115357    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:29.216288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:29.216252    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:29.372402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:29.372299    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:29.511426 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:29.511388    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:32.944293 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:32.944255    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:33.041649 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:33.041615    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:33.641679 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:33.641640    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:34.383499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.383458    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:44:34.387161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.387142    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.390011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.389987    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-a7ef4b-serving-cert\""
Sep 08 13:44:34.398061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.398036    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:44:34.413727 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.413695    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:34.462625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.462589    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.462812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.462634    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.462812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.462692    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.462812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.462732    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72d66\" (UniqueName: \"kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.478590 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.478502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:34.553165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.553124    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:44:34.564005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.563977    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.564154 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.564015    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.564154 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.564136    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.564272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.564170    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72d66\" (UniqueName: \"kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.564331 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:34.564300    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-serving-cert: secret "maas-api-e2e-inf-b-a7ef4b-serving-cert" not found
Sep 08 13:44:34.564384 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:34.564375    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls podName:1cee1059-f5bd-484d-a5c5-79a0f542a9d6 nodeName:}" failed. No retries permitted until 2026-09-08 13:44:35.064356004 +0000 UTC m=+961.282272308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls") pod "maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" (UID: "1cee1059-f5bd-484d-a5c5-79a0f542a9d6") : secret "maas-api-e2e-inf-b-a7ef4b-serving-cert" not found
Sep 08 13:44:34.564765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.564745    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.566707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.566688    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:34.573638 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:34.573613    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72d66\" (UniqueName: \"kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:35.017427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.017391    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:35.070075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.070039    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:35.072530 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.072501    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") pod \"maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:35.159361 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.159318    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:35.301868 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.301816    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:35.485664 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.485625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:35.593606 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:35.593577    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:44:35.595720 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:44:35.595690    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cee1059_f5bd_484d_a5c5_79a0f542a9d6.slice/crio-f5470c74276a9bd68feb816fd4baec2dd630a7b7349c93234bb3932521a8c6ab WatchSource:0}: Error finding container f5470c74276a9bd68feb816fd4baec2dd630a7b7349c93234bb3932521a8c6ab: Status 404 returned error can't find the container with id f5470c74276a9bd68feb816fd4baec2dd630a7b7349c93234bb3932521a8c6ab
Sep 08 13:44:36.285781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.285737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:36.350229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.350171    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" event={"ID":"1cee1059-f5bd-484d-a5c5-79a0f542a9d6","Type":"ContainerStarted","Data":"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"}
Sep 08 13:44:36.350408 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.350251    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:36.350408 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.350275    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" event={"ID":"1cee1059-f5bd-484d-a5c5-79a0f542a9d6","Type":"ContainerStarted","Data":"f5470c74276a9bd68feb816fd4baec2dd630a7b7349c93234bb3932521a8c6ab"}
Sep 08 13:44:36.372893 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.372841    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" podStartSLOduration=2.049976163 podStartE2EDuration="2.372823876s" totalImagesPullingTime="322.847713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:44:36.370652271 +0000 UTC m=+962.588568628" watchObservedRunningTime="2026-09-08 13:44:36.372823876 +0000 UTC m=+962.590740200"
Sep 08 13:44:36.649731 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.649635    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:36.785349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:36.785310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:37.376198 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:37.376156    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:38.046626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:38.046597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:38.805963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:38.805924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:40.337367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:40.337332    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:42.075507 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:42.075473    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:42.361966 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:42.361882    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:44:45.696727 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:45.696694    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:50.839884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:50.839843    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:44:52.777274 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:52.777238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:53.372955 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.372917    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:53.373204 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.373178    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" podUID="28991251-daef-4679-9d9a-258f27800d34" containerName="maas-api" containerID="cri-o://98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4" gracePeriod=30
Sep 08 13:44:53.401917 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.401881    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:53.480490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.480452    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:53.581880 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.581728    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:53.629921 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.629852    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:53.647409 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-476g4\" (UniqueName: \"kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4\") pod \"28991251-daef-4679-9d9a-258f27800d34\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") "
Sep 08 13:44:53.647544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca\") pod \"28991251-daef-4679-9d9a-258f27800d34\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") "
Sep 08 13:44:53.647544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647501    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs\") pod \"28991251-daef-4679-9d9a-258f27800d34\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") "
Sep 08 13:44:53.647544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647520    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls\") pod \"28991251-daef-4679-9d9a-258f27800d34\" (UID: \"28991251-daef-4679-9d9a-258f27800d34\") "
Sep 08 13:44:53.647817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647791    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28991251-daef-4679-9d9a-258f27800d34" (UID: "28991251-daef-4679-9d9a-258f27800d34"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:44:53.647903 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.647871    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28991251-daef-4679-9d9a-258f27800d34-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:44:53.649827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.649804    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28991251-daef-4679-9d9a-258f27800d34" (UID: "28991251-daef-4679-9d9a-258f27800d34"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:44:53.649932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.649913    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28991251-daef-4679-9d9a-258f27800d34" (UID: "28991251-daef-4679-9d9a-258f27800d34"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:44:53.649977 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.649918    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4" (OuterVolumeSpecName: "kube-api-access-476g4") pod "28991251-daef-4679-9d9a-258f27800d34" (UID: "28991251-daef-4679-9d9a-258f27800d34"). InnerVolumeSpecName "kube-api-access-476g4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:44:53.661399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.661369    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:53.748831 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.748795    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-476g4\" (UniqueName: \"kubernetes.io/projected/28991251-daef-4679-9d9a-258f27800d34-kube-api-access-476g4\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:44:53.748831 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.748827    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:44:53.749039 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.748843    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28991251-daef-4679-9d9a-258f27800d34-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:44:53.807321 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:53.807282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:54.377087 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.377055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:54.468492 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.468461    2583 generic.go:356] "Generic (PLEG): container finished" podID="28991251-daef-4679-9d9a-258f27800d34" containerID="98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4" exitCode=0
Sep 08 13:44:54.468722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.468500    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" event={"ID":"28991251-daef-4679-9d9a-258f27800d34","Type":"ContainerDied","Data":"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"}
Sep 08 13:44:54.468722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.468580    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd" event={"ID":"28991251-daef-4679-9d9a-258f27800d34","Type":"ContainerDied","Data":"b9325c3803320fb0362dbc37b8ba9c2f7a4566db2cf7ed6a593b30b6ff1a2dd2"}
Sep 08 13:44:54.468722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.468510    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"
Sep 08 13:44:54.468722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.468611    2583 scope.go:117] "RemoveContainer" containerID="98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"
Sep 08 13:44:54.479918 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.479897    2583 scope.go:117] "RemoveContainer" containerID="98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"
Sep 08 13:44:54.480247 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:54.480222    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4\": container with ID starting with 98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4 not found: ID does not exist" containerID="98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"
Sep 08 13:44:54.480301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.480260    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4"} err="failed to get container status \"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4\": rpc error: code = NotFound desc = could not find container \"98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4\": container with ID starting with 98bbf5fec2d855de99101d1d5d756e2d0221650d6985bdcd9065f624cdab3da4 not found: ID does not exist"
Sep 08 13:44:54.520117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.520088    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:54.523253 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.523211    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:54.540412 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.540383    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b415db2-7c46c8d697-5bzgd"]
Sep 08 13:44:54.634597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.634567    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:54.841102 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:54.841014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:55.026346 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:55.026304    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:55.480608 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:55.480497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:56.233017 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:56.232975    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28991251-daef-4679-9d9a-258f27800d34" path="/var/lib/kubelet/pods/28991251-daef-4679-9d9a-258f27800d34/volumes"
Sep 08 13:44:57.703280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.703240    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:57.811789 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.811753    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:44:57.812368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.812338    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28991251-daef-4679-9d9a-258f27800d34" containerName="maas-api"
Sep 08 13:44:57.812368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.812365    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="28991251-daef-4679-9d9a-258f27800d34" containerName="maas-api"
Sep 08 13:44:57.812584 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.812496    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="28991251-daef-4679-9d9a-258f27800d34" containerName="maas-api"
Sep 08 13:44:57.813948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.813919    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:57.817403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.817386    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.820153 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.820133    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-fed6eaa3-serving-cert\""
Sep 08 13:44:57.831253 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.831225    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:44:57.880045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.880009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:57.892193 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.892156    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmq5b\" (UniqueName: \"kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.892193 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.892194    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.892422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.892220    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.892422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.892340    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.994005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.993903    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmq5b\" (UniqueName: \"kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.994005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.993945    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.994005 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.993976    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.994335 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.994031    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.994335 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:57.994136    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-serving-cert: secret "maas-api-e2e-migrate-fed6eaa3-serving-cert" not found
Sep 08 13:44:57.994335 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:44:57.994231    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls podName:21ea2dce-d8c4-4c8c-9112-edcd9b1081bc nodeName:}" failed. No retries permitted until 2026-09-08 13:44:58.494207996 +0000 UTC m=+984.712124313 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls") pod "maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" (UID: "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc") : secret "maas-api-e2e-migrate-fed6eaa3-serving-cert" not found
Sep 08 13:44:57.994725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.994702    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.996757 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.996737    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:57.998803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:57.998777    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:58.002954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.002931    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmq5b\" (UniqueName: \"kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:58.227433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.227368    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:58.364320 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.364283    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:58.465441 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.465394    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:44:58.499461 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.499433    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:58.502200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.502168    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") pod \"maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:58.730695 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:58.730588    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:44:59.009914 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:59.009883    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:44:59.011347 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:44:59.011306    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21ea2dce_d8c4_4c8c_9112_edcd9b1081bc.slice/crio-48bd783558b187c8a9692e776a727e0070dba81cfa7e8c9b2843ab933cf8a9a6 WatchSource:0}: Error finding container 48bd783558b187c8a9692e776a727e0070dba81cfa7e8c9b2843ab933cf8a9a6: Status 404 returned error can't find the container with id 48bd783558b187c8a9692e776a727e0070dba81cfa7e8c9b2843ab933cf8a9a6
Sep 08 13:44:59.508879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:44:59.508843    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" event={"ID":"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc","Type":"ContainerStarted","Data":"48bd783558b187c8a9692e776a727e0070dba81cfa7e8c9b2843ab933cf8a9a6"}
Sep 08 13:45:00.516803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:00.516767    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" event={"ID":"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc","Type":"ContainerStarted","Data":"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"}
Sep 08 13:45:00.517301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:00.516922    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:45:00.535612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:00.535527    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" podStartSLOduration=2.897172235 podStartE2EDuration="3.53550562s" totalImagesPullingTime="638.333385ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:44:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:45:00.535009141 +0000 UTC m=+986.752925478" watchObservedRunningTime="2026-09-08 13:45:00.53550562 +0000 UTC m=+986.753421948"
Sep 08 13:45:06.529984 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:06.529951    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:45:08.546701 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:08.546662    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:45:15.559346 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:15.559308    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:15.635623 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:15.635575    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.263949 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.263908    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:45:29.264491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.264166    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" podUID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" containerName="maas-api" containerID="cri-o://0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73" gracePeriod=30
Sep 08 13:45:29.295806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.295765    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.362617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.362577    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.429329 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.429278    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.537270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.537243    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:45:29.539692 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.539665    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.619170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.619137    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:29.621988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.621969    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs\") pod \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") "
Sep 08 13:45:29.622129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.622049    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmq5b\" (UniqueName: \"kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b\") pod \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") "
Sep 08 13:45:29.622129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.622094    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") pod \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") "
Sep 08 13:45:29.622226 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.622159    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca\") pod \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\" (UID: \"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc\") "
Sep 08 13:45:29.622685 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.622646    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" (UID: "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:45:29.624290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.624257    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" (UID: "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:45:29.624290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.624273    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b" (OuterVolumeSpecName: "kube-api-access-dmq5b") pod "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" (UID: "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc"). InnerVolumeSpecName "kube-api-access-dmq5b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:45:29.624428 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.624298    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" (UID: "21ea2dce-d8c4-4c8c-9112-edcd9b1081bc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:45:29.704795 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.704763    2583 generic.go:356] "Generic (PLEG): container finished" podID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" containerID="0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73" exitCode=0
Sep 08 13:45:29.704994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.704824    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"
Sep 08 13:45:29.704994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.704869    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" event={"ID":"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc","Type":"ContainerDied","Data":"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"}
Sep 08 13:45:29.704994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.704938    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz" event={"ID":"21ea2dce-d8c4-4c8c-9112-edcd9b1081bc","Type":"ContainerDied","Data":"48bd783558b187c8a9692e776a727e0070dba81cfa7e8c9b2843ab933cf8a9a6"}
Sep 08 13:45:29.704994 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.704968    2583 scope.go:117] "RemoveContainer" containerID="0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"
Sep 08 13:45:29.715843 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.715822    2583 scope.go:117] "RemoveContainer" containerID="0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"
Sep 08 13:45:29.716226 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:45:29.716202    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73\": container with ID starting with 0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73 not found: ID does not exist" containerID="0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"
Sep 08 13:45:29.716354 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.716232    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73"} err="failed to get container status \"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73\": rpc error: code = NotFound desc = could not find container \"0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73\": container with ID starting with 0eeb28d30466cb433b11347d107b5292c8a24afb8c15d1b692a77e5360abed73 not found: ID does not exist"
Sep 08 13:45:29.723216 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.723194    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:45:29.723216 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.723213    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:45:29.723402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.723223    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmq5b\" (UniqueName: \"kubernetes.io/projected/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-kube-api-access-dmq5b\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:45:29.723402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.723231    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:45:29.743745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.743713    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:45:29.757319 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.757292    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-fed6eaa3-678d6c8c6f-pz6jz"]
Sep 08 13:45:29.821589 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:29.821467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:30.237859 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:30.237774    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" path="/var/lib/kubelet/pods/21ea2dce-d8c4-4c8c-9112-edcd9b1081bc/volumes"
Sep 08 13:45:30.583017 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:30.582975    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:30.683240 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:30.683208    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:31.270766 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:31.270733    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:32.716272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:32.716236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:32.813834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:32.813797    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:33.540066 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.540039    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:33.871316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.871231    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:45:33.871737 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.871722    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" containerName="maas-api"
Sep 08 13:45:33.871782 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.871739    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" containerName="maas-api"
Sep 08 13:45:33.871822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.871817    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="21ea2dce-d8c4-4c8c-9112-edcd9b1081bc" containerName="maas-api"
Sep 08 13:45:33.875229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.875207    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:33.881725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.881455    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-515aa5b9-serving-cert\""
Sep 08 13:45:33.903307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.903272    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:45:33.938146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.938110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:33.966523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.966481    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:33.966755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.966530    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5nkf\" (UniqueName: \"kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:33.966755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.966626    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:33.966848 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:33.966792    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.040012 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.039979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:34.067473 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.067434    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5nkf\" (UniqueName: \"kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.067689 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.067488    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.067689 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.067619    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.067808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.067696    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.067853 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:45:34.067802    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-serving-cert: secret "maas-api-e2e-ait-515aa5b9-serving-cert" not found
Sep 08 13:45:34.067912 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:45:34.067899    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls podName:9a28d622-5a56-4f8e-856a-fa45266e9cdd nodeName:}" failed. No retries permitted until 2026-09-08 13:45:34.567864021 +0000 UTC m=+1020.785780323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls") pod "maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" (UID: "9a28d622-5a56-4f8e-856a-fa45266e9cdd") : secret "maas-api-e2e-ait-515aa5b9-serving-cert" not found
Sep 08 13:45:34.068328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.068311    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.070209 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.070187    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.076975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.076951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5nkf\" (UniqueName: \"kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.351389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.351347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:34.573672 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.573635    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.576258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.576230    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") pod \"maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.789340 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.789297    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:34.897829 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.897720    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:34.993606 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:34.993534    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:35.067035 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.067003    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:45:35.069325 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:45:35.069290    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a28d622_5a56_4f8e_856a_fa45266e9cdd.slice/crio-bb2cfe988c08320c04c9604a5870697d8778b610dd6225f93e6f94b363b1dfc7 WatchSource:0}: Error finding container bb2cfe988c08320c04c9604a5870697d8778b610dd6225f93e6f94b363b1dfc7: Status 404 returned error can't find the container with id bb2cfe988c08320c04c9604a5870697d8778b610dd6225f93e6f94b363b1dfc7
Sep 08 13:45:35.070799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.070780    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:45:35.745609 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.745518    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" event={"ID":"9a28d622-5a56-4f8e-856a-fa45266e9cdd","Type":"ContainerStarted","Data":"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"}
Sep 08 13:45:35.745609 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.745603    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:35.745875 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.745630    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" event={"ID":"9a28d622-5a56-4f8e-856a-fa45266e9cdd","Type":"ContainerStarted","Data":"bb2cfe988c08320c04c9604a5870697d8778b610dd6225f93e6f94b363b1dfc7"}
Sep 08 13:45:35.765147 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.765091    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" podStartSLOduration=2.279589044 podStartE2EDuration="2.765074859s" totalImagesPullingTime="485.485815ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:45:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:45:35.762413957 +0000 UTC m=+1021.980330321" watchObservedRunningTime="2026-09-08 13:45:35.765074859 +0000 UTC m=+1021.982991200"
Sep 08 13:45:35.874633 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:35.874502    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:37.342882 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:37.342847    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:45:41.757810 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:41.757770    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:45:56.138523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:56.138489    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:45:56.555067 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:56.555031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:45:58.836774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:45:58.836743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:04.467891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.467846    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:46:04.468408 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.468107    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" podUID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" containerName="maas-api" containerID="cri-o://c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d" gracePeriod=30
Sep 08 13:46:04.499901 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.499868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:04.598981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.598947    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:04.687638 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.687597    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:04.731432 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.731370    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:46:04.764812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.764778    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:04.881504 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.881464    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs\") pod \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") "
Sep 08 13:46:04.881712 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.881524    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5nkf\" (UniqueName: \"kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf\") pod \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") "
Sep 08 13:46:04.881712 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.881581    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") pod \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") "
Sep 08 13:46:04.881712 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.881692    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca\") pod \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\" (UID: \"9a28d622-5a56-4f8e-856a-fa45266e9cdd\") "
Sep 08 13:46:04.882435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.882403    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9a28d622-5a56-4f8e-856a-fa45266e9cdd" (UID: "9a28d622-5a56-4f8e-856a-fa45266e9cdd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:04.883981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.883946    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf" (OuterVolumeSpecName: "kube-api-access-p5nkf") pod "9a28d622-5a56-4f8e-856a-fa45266e9cdd" (UID: "9a28d622-5a56-4f8e-856a-fa45266e9cdd"). InnerVolumeSpecName "kube-api-access-p5nkf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:46:04.883981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.883972    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9a28d622-5a56-4f8e-856a-fa45266e9cdd" (UID: "9a28d622-5a56-4f8e-856a-fa45266e9cdd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:04.884211 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.884191    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9a28d622-5a56-4f8e-856a-fa45266e9cdd" (UID: "9a28d622-5a56-4f8e-856a-fa45266e9cdd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:04.938157 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.938121    2583 generic.go:356] "Generic (PLEG): container finished" podID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" containerID="c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d" exitCode=0
Sep 08 13:46:04.938330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.938166    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"
Sep 08 13:46:04.938330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.938210    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" event={"ID":"9a28d622-5a56-4f8e-856a-fa45266e9cdd","Type":"ContainerDied","Data":"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"}
Sep 08 13:46:04.938330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.938268    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2" event={"ID":"9a28d622-5a56-4f8e-856a-fa45266e9cdd","Type":"ContainerDied","Data":"bb2cfe988c08320c04c9604a5870697d8778b610dd6225f93e6f94b363b1dfc7"}
Sep 08 13:46:04.938330 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.938292    2583 scope.go:117] "RemoveContainer" containerID="c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"
Sep 08 13:46:04.958631 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.958608    2583 scope.go:117] "RemoveContainer" containerID="c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"
Sep 08 13:46:04.958949 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:46:04.958928    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d\": container with ID starting with c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d not found: ID does not exist" containerID="c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"
Sep 08 13:46:04.959015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.958957    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d"} err="failed to get container status \"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d\": rpc error: code = NotFound desc = could not find container \"c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d\": container with ID starting with c3720164811806505ffa765c9ab4bda246b286df8834b6adc3af50be9b39a97d not found: ID does not exist"
Sep 08 13:46:04.969077 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.969052    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:46:04.972424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.972397    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-515aa5b9-7c6df496d-hccj2"]
Sep 08 13:46:04.983200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.983121    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9a28d622-5a56-4f8e-856a-fa45266e9cdd-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:04.983200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.983149    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:04.983200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.983160    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5nkf\" (UniqueName: \"kubernetes.io/projected/9a28d622-5a56-4f8e-856a-fa45266e9cdd-kube-api-access-p5nkf\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:04.983200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:04.983168    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9a28d622-5a56-4f8e-856a-fa45266e9cdd-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:05.393184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:05.393149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:05.939523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:05.939467    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:06.232316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:06.232228    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" path="/var/lib/kubelet/pods/9a28d622-5a56-4f8e-856a-fa45266e9cdd/volumes"
Sep 08 13:46:06.389177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:06.389144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:06.725394 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:06.725354    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:07.172889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:07.172856    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:09.013919 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:09.013878    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:09.111403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:09.111370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.130443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.130404    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:10.131100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.131080    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" containerName="maas-api"
Sep 08 13:46:10.131191 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.131103    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" containerName="maas-api"
Sep 08 13:46:10.131269 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.131256    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a28d622-5a56-4f8e-856a-fa45266e9cdd" containerName="maas-api"
Sep 08 13:46:10.136411 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.136390    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.138669 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.138650    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2781f063-serving-cert\""
Sep 08 13:46:10.159883 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.159831    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:10.173306 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.173275    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.232119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.232092    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.232275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.232139    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.232275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.232160    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4lxp\" (UniqueName: \"kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.232275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.232234    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.273346 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.273311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.332803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.332764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.332964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.332836    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.332964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.332857    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4lxp\" (UniqueName: \"kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.332964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.332936    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.333598 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.333577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.335404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.335384    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.335511 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.335451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.341310 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.341284    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4lxp\" (UniqueName: \"kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp\") pod \"maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.460169 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.460076    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:10.593201 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.593158    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.702614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.702569    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.739349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.739323    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:10.740214 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:46:10.740174    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60bb794c_e6ab_4fe5_9f2e_7b8a81e47b21.slice/crio-c578586b3ccbbbd29479e016cbf4272bef4bbfcb028cea21ebeb569a55fa7d34 WatchSource:0}: Error finding container c578586b3ccbbbd29479e016cbf4272bef4bbfcb028cea21ebeb569a55fa7d34: Status 404 returned error can't find the container with id c578586b3ccbbbd29479e016cbf4272bef4bbfcb028cea21ebeb569a55fa7d34
Sep 08 13:46:10.790563 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.790515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:10.979212 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:10.979175    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" event={"ID":"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21","Type":"ContainerStarted","Data":"c578586b3ccbbbd29479e016cbf4272bef4bbfcb028cea21ebeb569a55fa7d34"}
Sep 08 13:46:11.449195 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:11.449163    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:46:11.988133 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:11.988088    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" event={"ID":"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21","Type":"ContainerStarted","Data":"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"}
Sep 08 13:46:11.988342 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:11.988158    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:12.007256 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:12.007194    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" podStartSLOduration=1.545228422 podStartE2EDuration="2.007173888s" totalImagesPullingTime="461.945466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:46:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:46:12.006768421 +0000 UTC m=+1058.224684797" watchObservedRunningTime="2026-09-08 13:46:12.007173888 +0000 UTC m=+1058.225090214"
Sep 08 13:46:15.392960 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:15.392915    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:15.964048 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:15.964016    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:46:15.964303 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:15.964280    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" podUID="8b4f0440-2f82-4470-a2be-c44f62aa5412" containerName="maas-api" containerID="cri-o://7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142" gracePeriod=30
Sep 08 13:46:16.003120 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.003089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:16.158081 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.158002    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:16.249643 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.249618    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:16.260704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.260684    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:46:16.284860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.284831    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs\") pod \"8b4f0440-2f82-4470-a2be-c44f62aa5412\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") "
Sep 08 13:46:16.285053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.284887    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") pod \"8b4f0440-2f82-4470-a2be-c44f62aa5412\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") "
Sep 08 13:46:16.285053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.284966    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca\") pod \"8b4f0440-2f82-4470-a2be-c44f62aa5412\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") "
Sep 08 13:46:16.285053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.285004    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqnmw\" (UniqueName: \"kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw\") pod \"8b4f0440-2f82-4470-a2be-c44f62aa5412\" (UID: \"8b4f0440-2f82-4470-a2be-c44f62aa5412\") "
Sep 08 13:46:16.285913 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.285863    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8b4f0440-2f82-4470-a2be-c44f62aa5412" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:16.287827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.287785    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw" (OuterVolumeSpecName: "kube-api-access-xqnmw") pod "8b4f0440-2f82-4470-a2be-c44f62aa5412" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412"). InnerVolumeSpecName "kube-api-access-xqnmw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:46:16.289270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.288753    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8b4f0440-2f82-4470-a2be-c44f62aa5412" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:16.289270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.289035    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8b4f0440-2f82-4470-a2be-c44f62aa5412" (UID: "8b4f0440-2f82-4470-a2be-c44f62aa5412"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:16.386455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.386417    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b4f0440-2f82-4470-a2be-c44f62aa5412-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.386455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.386447    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqnmw\" (UniqueName: \"kubernetes.io/projected/8b4f0440-2f82-4470-a2be-c44f62aa5412-kube-api-access-xqnmw\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.386455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.386457    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.386716 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.386467    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b4f0440-2f82-4470-a2be-c44f62aa5412-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:16.624692 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:16.624662    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:17.022466 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.022432    2583 generic.go:356] "Generic (PLEG): container finished" podID="8b4f0440-2f82-4470-a2be-c44f62aa5412" containerID="7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142" exitCode=0
Sep 08 13:46:17.022700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.022480    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"
Sep 08 13:46:17.022700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.022520    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" event={"ID":"8b4f0440-2f82-4470-a2be-c44f62aa5412","Type":"ContainerDied","Data":"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"}
Sep 08 13:46:17.022700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.022590    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh" event={"ID":"8b4f0440-2f82-4470-a2be-c44f62aa5412","Type":"ContainerDied","Data":"198460756e8f20b332eed194b8aa0e90f4d211260215087e30f0c7ecacbe6c80"}
Sep 08 13:46:17.022700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.022620    2583 scope.go:117] "RemoveContainer" containerID="7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"
Sep 08 13:46:17.033052 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.033028    2583 scope.go:117] "RemoveContainer" containerID="7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"
Sep 08 13:46:17.033380 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:46:17.033360    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142\": container with ID starting with 7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142 not found: ID does not exist" containerID="7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"
Sep 08 13:46:17.033459 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.033390    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142"} err="failed to get container status \"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142\": rpc error: code = NotFound desc = could not find container \"7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142\": container with ID starting with 7bd7b70eb16a7dc6b40e25a5d7da24d72bb4e3f0145eff2fb2dd731c7e21e142 not found: ID does not exist"
Sep 08 13:46:17.046950 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.046912    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:46:17.051352 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:17.051331    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-b719d2-69f9b65457-lmwdh"]
Sep 08 13:46:18.000821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:18.000792    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:18.232716 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:18.232680    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b4f0440-2f82-4470-a2be-c44f62aa5412" path="/var/lib/kubelet/pods/8b4f0440-2f82-4470-a2be-c44f62aa5412/volumes"
Sep 08 13:46:21.652491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:21.652451    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:27.701179 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:27.700952    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:28.143600 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:28.143561    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:46:31.189690 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:31.189654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:32.164748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:32.164717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:37.563618 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:37.563579    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:38.450523 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:38.450489    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:46:40.238527 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.238481    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:40.720524 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.720411    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:40.720734 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.720695    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" podUID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" containerName="maas-api" containerID="cri-o://20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca" gracePeriod=30
Sep 08 13:46:40.753009 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.752979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:40.826632 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.826535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:40.899750 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.899716    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:40.987137 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:40.987076    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:41.147802 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.147770    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs\") pod \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") "
Sep 08 13:46:41.148004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.147862    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca\") pod \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") "
Sep 08 13:46:41.148004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.147898    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls\") pod \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") "
Sep 08 13:46:41.148004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.147931    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j4lxp\" (UniqueName: \"kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp\") pod \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\" (UID: \"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21\") "
Sep 08 13:46:41.148297 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.148269    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" (UID: "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:46:41.150371 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.150343    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" (UID: "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:41.150898 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.150876    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" (UID: "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:46:41.151034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.151018    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp" (OuterVolumeSpecName: "kube-api-access-j4lxp") pod "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" (UID: "60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21"). InnerVolumeSpecName "kube-api-access-j4lxp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:46:41.174921 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.174888    2583 generic.go:356] "Generic (PLEG): container finished" podID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" containerID="20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca" exitCode=0
Sep 08 13:46:41.175075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.174933    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"
Sep 08 13:46:41.175075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.174952    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" event={"ID":"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21","Type":"ContainerDied","Data":"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"}
Sep 08 13:46:41.175075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.174999    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6" event={"ID":"60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21","Type":"ContainerDied","Data":"c578586b3ccbbbd29479e016cbf4272bef4bbfcb028cea21ebeb569a55fa7d34"}
Sep 08 13:46:41.175075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.175022    2583 scope.go:117] "RemoveContainer" containerID="20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"
Sep 08 13:46:41.186584 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.186532    2583 scope.go:117] "RemoveContainer" containerID="20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"
Sep 08 13:46:41.186894 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:46:41.186874    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca\": container with ID starting with 20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca not found: ID does not exist" containerID="20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"
Sep 08 13:46:41.186942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.186904    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca"} err="failed to get container status \"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca\": rpc error: code = NotFound desc = could not find container \"20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca\": container with ID starting with 20b0465b07a7765183c3bfcdfdb94970816e858f57b0c6c7ebde876b09fac3ca not found: ID does not exist"
Sep 08 13:46:41.201011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.200981    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:41.205114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.205085    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2781f063-74dd4c676c-hl4s6"]
Sep 08 13:46:41.249051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.248952    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:41.249051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.248988    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:41.249051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.249004    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:41.249051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.249024    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j4lxp\" (UniqueName: \"kubernetes.io/projected/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21-kube-api-access-j4lxp\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:46:41.381756 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.381715    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:41.482144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.482110    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:41.892075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.892041    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:41.996358 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:41.996311    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:42.099455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:42.099422    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:42.231887 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:42.231800    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" path="/var/lib/kubelet/pods/60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21/volumes"
Sep 08 13:46:44.529647 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:44.529611    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:53.959629 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:53.959586    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:55.235061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:55.235023    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:46:55.755768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:55.755706    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:58.930546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:58.930516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:59.036546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:59.036516    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:59.542752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:59.542714    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:59.863463 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:59.863333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:46:59.960943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:46:59.960896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:00.345699 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:00.345668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:00.452168 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:00.452133    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:00.527384 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:00.527349    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:00.627308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:00.627217    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:01.122889 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:01.122847    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:02.824321 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:02.824286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:02.985065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:02.985009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:04.028659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.028624    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:04.029108 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029093    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" containerName="maas-api"
Sep 08 13:47:04.029166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029112    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" containerName="maas-api"
Sep 08 13:47:04.029166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029128    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b4f0440-2f82-4470-a2be-c44f62aa5412" containerName="maas-api"
Sep 08 13:47:04.029166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029134    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b4f0440-2f82-4470-a2be-c44f62aa5412" containerName="maas-api"
Sep 08 13:47:04.029283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029235    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b4f0440-2f82-4470-a2be-c44f62aa5412" containerName="maas-api"
Sep 08 13:47:04.029283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.029245    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="60bb794c-e6ab-4fe5-9f2e-7b8a81e47b21" containerName="maas-api"
Sep 08 13:47:04.032187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.032162    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.035196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.035171    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-d60b5a-serving-cert\""
Sep 08 13:47:04.044483 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.044456    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:04.170079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.170042    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:04.184347 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.184302    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.184503 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.184407    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.184569 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.184536    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klvj5\" (UniqueName: \"kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.184622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.184606    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.284115 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.284025    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:04.286219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.286190    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.286361 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.286339    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.286525 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.286503    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klvj5\" (UniqueName: \"kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.286783 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.286764    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.287097 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.287071    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.289323 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.289297    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.289704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.289686    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.295091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.295067    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klvj5\" (UniqueName: \"kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5\") pod \"maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.345458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.345424    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:04.573279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.573184    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:04.629957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.629930    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:04.631363 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:47:04.631334    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06118a41_7939_412c_8f34_83970d4c9147.slice/crio-5de9ff533dfa9c3b6eac2603baf58453641021b28d87ad1d052f10e5a2e4f22d WatchSource:0}: Error finding container 5de9ff533dfa9c3b6eac2603baf58453641021b28d87ad1d052f10e5a2e4f22d: Status 404 returned error can't find the container with id 5de9ff533dfa9c3b6eac2603baf58453641021b28d87ad1d052f10e5a2e4f22d
Sep 08 13:47:04.672738 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:04.672704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:05.332530 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:05.332476    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" event={"ID":"06118a41-7939-412c-8f34-83970d4c9147","Type":"ContainerStarted","Data":"e9d7e417b25d62d6d46f1f9517016166fd6edd7092ecd6fecf9f9bb3f4aea168"}
Sep 08 13:47:05.332955 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:05.332599    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" event={"ID":"06118a41-7939-412c-8f34-83970d4c9147","Type":"ContainerStarted","Data":"5de9ff533dfa9c3b6eac2603baf58453641021b28d87ad1d052f10e5a2e4f22d"}
Sep 08 13:47:05.332955 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:05.332634    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:05.356416 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:05.356368    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" podStartSLOduration=1.978417328 podStartE2EDuration="2.356350904s" totalImagesPullingTime="377.933576ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:47:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:47:05.355264151 +0000 UTC m=+1111.573180488" watchObservedRunningTime="2026-09-08 13:47:05.356350904 +0000 UTC m=+1111.574267231"
Sep 08 13:47:06.546171 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:06.546105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:06.646495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:06.646455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:08.947147 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:08.947109    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:47:11.344281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:11.344252    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:23.138818 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:23.138785    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:47:27.064151 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:27.064083    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:27.164109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:27.164070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:27.844198 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:27.844165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:34.679882 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:34.679844    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:34.779862 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:34.779828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:36.257360 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:36.257328    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:40.438414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:40.438364    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:47:45.683998 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:45.683957    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:49.453973 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:49.453917    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:55.314259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:55.314218    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:55.384424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:55.384348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:55.471272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:55.471234    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:55.573091 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:55.572998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:56.528846 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.528808    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:56.529320 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.529031    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" podUID="06118a41-7939-412c-8f34-83970d4c9147" containerName="maas-api" containerID="cri-o://e9d7e417b25d62d6d46f1f9517016166fd6edd7092ecd6fecf9f9bb3f4aea168" gracePeriod=30
Sep 08 13:47:56.565160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.565117    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:56.661599 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.661534    2583 generic.go:356] "Generic (PLEG): container finished" podID="06118a41-7939-412c-8f34-83970d4c9147" containerID="e9d7e417b25d62d6d46f1f9517016166fd6edd7092ecd6fecf9f9bb3f4aea168" exitCode=0
Sep 08 13:47:56.661756 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.661617    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" event={"ID":"06118a41-7939-412c-8f34-83970d4c9147","Type":"ContainerDied","Data":"e9d7e417b25d62d6d46f1f9517016166fd6edd7092ecd6fecf9f9bb3f4aea168"}
Sep 08 13:47:56.666135 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.666105    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:56.762639 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.762605    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:56.785698 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.785633    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:56.834764 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.834730    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs\") pod \"06118a41-7939-412c-8f34-83970d4c9147\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") "
Sep 08 13:47:56.834974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.834841    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls\") pod \"06118a41-7939-412c-8f34-83970d4c9147\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") "
Sep 08 13:47:56.834974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.834863    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klvj5\" (UniqueName: \"kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5\") pod \"06118a41-7939-412c-8f34-83970d4c9147\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") "
Sep 08 13:47:56.834974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.834912    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca\") pod \"06118a41-7939-412c-8f34-83970d4c9147\" (UID: \"06118a41-7939-412c-8f34-83970d4c9147\") "
Sep 08 13:47:56.835355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.835330    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06118a41-7939-412c-8f34-83970d4c9147" (UID: "06118a41-7939-412c-8f34-83970d4c9147"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:47:56.837181 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.837157    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06118a41-7939-412c-8f34-83970d4c9147" (UID: "06118a41-7939-412c-8f34-83970d4c9147"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:47:56.837289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.837199    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06118a41-7939-412c-8f34-83970d4c9147" (UID: "06118a41-7939-412c-8f34-83970d4c9147"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:47:56.837289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.837237    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5" (OuterVolumeSpecName: "kube-api-access-klvj5") pod "06118a41-7939-412c-8f34-83970d4c9147" (UID: "06118a41-7939-412c-8f34-83970d4c9147"). InnerVolumeSpecName "kube-api-access-klvj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:47:56.862838 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.862806    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:47:56.936187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.936150    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:47:56.936187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.936185    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klvj5\" (UniqueName: \"kubernetes.io/projected/06118a41-7939-412c-8f34-83970d4c9147-kube-api-access-klvj5\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:47:56.936187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.936196    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06118a41-7939-412c-8f34-83970d4c9147-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:47:56.936187 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:56.936205    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06118a41-7939-412c-8f34-83970d4c9147-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:47:57.669707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:57.669668    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq" event={"ID":"06118a41-7939-412c-8f34-83970d4c9147","Type":"ContainerDied","Data":"5de9ff533dfa9c3b6eac2603baf58453641021b28d87ad1d052f10e5a2e4f22d"}
Sep 08 13:47:57.669707 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:57.669702    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"
Sep 08 13:47:57.670203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:57.669759    2583 scope.go:117] "RemoveContainer" containerID="e9d7e417b25d62d6d46f1f9517016166fd6edd7092ecd6fecf9f9bb3f4aea168"
Sep 08 13:47:57.693773 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:57.693738    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:57.698051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:57.698026    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-d60b5a-79cf94c7d5-hrtrq"]
Sep 08 13:47:58.231364 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:58.231326    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06118a41-7939-412c-8f34-83970d4c9147" path="/var/lib/kubelet/pods/06118a41-7939-412c-8f34-83970d4c9147/volumes"
Sep 08 13:47:58.621477 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:47:58.621442    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:00.034908 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:00.034876    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:48:03.362679 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:03.362634    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:03.464020 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:03.463979    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:03.652847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:03.652760    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:04.169224 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:04.169190    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:04.498060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:04.497967    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:04.774033 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:04.773998    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:05.527927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:05.527894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:05.635923 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:05.635880    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:05.784162 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:05.784086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:06.081959 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:06.081861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:06.941584 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:06.941523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:13.180104 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:13.180056    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:13.244228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:13.244193    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:14.834480 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:14.834444    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:14.935458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:14.935406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:15.085443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:15.085348    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:16.239702 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:16.239671    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:18.540524 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:18.540485    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:48:22.096151 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:22.096114    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:22.195937 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:22.195897    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:22.826678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:22.826644    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:28.964025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:28.963989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:29.278803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:29.278759    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:29.378698 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:29.378664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:29.540701 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:29.540607    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:29.637568 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:29.637524    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:32.461470 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:32.461435    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:32.560155 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:32.560120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:33.440587 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.440518    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:48:33.829606 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.829570    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:48:33.830036 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.830022    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06118a41-7939-412c-8f34-83970d4c9147" containerName="maas-api"
Sep 08 13:48:33.830080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.830038    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="06118a41-7939-412c-8f34-83970d4c9147" containerName="maas-api"
Sep 08 13:48:33.830119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.830111    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="06118a41-7939-412c-8f34-83970d4c9147" containerName="maas-api"
Sep 08 13:48:33.832906 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.832889    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:33.835408 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.835388    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-f8154283-serving-cert\""
Sep 08 13:48:33.872382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.872345    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:48:33.889376 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.889331    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:33.923866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.923834    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:33.924076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.923886    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:33.924076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.923924    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npldg\" (UniqueName: \"kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:33.924076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.924003    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:33.993272 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:33.993238    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:34.024509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.024470    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npldg\" (UniqueName: \"kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.024728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.024602    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.024728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.024662    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.024728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.024711    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.024924 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:48:34.024838    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-serving-cert: secret "maas-api-e2e-mt-f8154283-serving-cert" not found
Sep 08 13:48:34.024924 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:48:34.024912    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls podName:468d74a0-bbf3-42b9-972e-4eda51ad3501 nodeName:}" failed. No retries permitted until 2026-09-08 13:48:34.52489059 +0000 UTC m=+1200.742806894 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls") pod "maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" (UID: "468d74a0-bbf3-42b9-972e-4eda51ad3501") : secret "maas-api-e2e-mt-f8154283-serving-cert" not found
Sep 08 13:48:34.025506 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.025429    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.027790 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.027765    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.035028 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.035004    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npldg\" (UniqueName: \"kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.213974 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.213877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:34.290412 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.290382    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:48:34.293280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.293259    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:48:34.392699 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.392596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:34.484642 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.484535    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:34.529422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.529390    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.532095 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.532071    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") pod \"maas-api-e2e-mt-f8154283-9d69b944c-4ddx6\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:34.743650 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:34.743537    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:35.022316 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:35.022289    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:48:35.024176 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:48:35.024151    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod468d74a0_bbf3_42b9_972e_4eda51ad3501.slice/crio-485d29868b0d98559cb9a263eae8277ae52c7922e184ec09abe92beed54d95db WatchSource:0}: Error finding container 485d29868b0d98559cb9a263eae8277ae52c7922e184ec09abe92beed54d95db: Status 404 returned error can't find the container with id 485d29868b0d98559cb9a263eae8277ae52c7922e184ec09abe92beed54d95db
Sep 08 13:48:35.922934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:35.922896    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" event={"ID":"468d74a0-bbf3-42b9-972e-4eda51ad3501","Type":"ContainerStarted","Data":"485d29868b0d98559cb9a263eae8277ae52c7922e184ec09abe92beed54d95db"}
Sep 08 13:48:37.937237 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:37.937193    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" event={"ID":"468d74a0-bbf3-42b9-972e-4eda51ad3501","Type":"ContainerStarted","Data":"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"}
Sep 08 13:48:37.937792 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:37.937419    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:37.956445 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:37.956394    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" podStartSLOduration=3.073467652 podStartE2EDuration="4.956377993s" totalImagesPullingTime="1.882910341s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:48:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:48:37.953975548 +0000 UTC m=+1204.171891874" watchObservedRunningTime="2026-09-08 13:48:37.956377993 +0000 UTC m=+1204.174294357"
Sep 08 13:48:43.950809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:43.950771    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:48:45.006060 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:45.006014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:46.077704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:46.077672    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:46.212698 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:46.212664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:46.486661 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:46.486566    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:46.622100 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:46.622068    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:47.307453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:47.307417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:47.516397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:47.516367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:48.949799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:48.949761    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:48:51.695672 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:51.695641    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:54.807562 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:54.807507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:57.536701 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:57.536668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:57.634856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:57.634818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:58.091817 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.091783    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:48:58.092043 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.092013    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" podUID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" containerName="maas-api" containerID="cri-o://bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf" gracePeriod=30
Sep 08 13:48:58.118486 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.118454    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:48:58.369876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.369850    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:48:58.382367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.382327    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls\") pod \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") "
Sep 08 13:48:58.382571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.382401    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs\") pod \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") "
Sep 08 13:48:58.382571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.382433    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chwb8\" (UniqueName: \"kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8\") pod \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") "
Sep 08 13:48:58.382571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.382465    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca\") pod \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\" (UID: \"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3\") "
Sep 08 13:48:58.383126 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.383100    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" (UID: "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:48:58.384769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.384740    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" (UID: "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:48:58.385220 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.385196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" (UID: "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:48:58.385315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.385216    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8" (OuterVolumeSpecName: "kube-api-access-chwb8") pod "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" (UID: "4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3"). InnerVolumeSpecName "kube-api-access-chwb8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:48:58.483739 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.483692    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.483739 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.483729    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.483739 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.483741    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chwb8\" (UniqueName: \"kubernetes.io/projected/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-kube-api-access-chwb8\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:48:58.484011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:58.483755    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:48:59.074258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.074215    2583 generic.go:356] "Generic (PLEG): container finished" podID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" containerID="bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf" exitCode=0
Sep 08 13:48:59.074781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.074261    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"
Sep 08 13:48:59.074781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.074305    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" event={"ID":"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3","Type":"ContainerDied","Data":"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"}
Sep 08 13:48:59.074781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.074363    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m" event={"ID":"4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3","Type":"ContainerDied","Data":"08722ae8240fe221c0ee11985ad723abc6f9bee4f09bb746bf2d5557bc6150c3"}
Sep 08 13:48:59.074781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.074390    2583 scope.go:117] "RemoveContainer" containerID="bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"
Sep 08 13:48:59.084780 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.084761    2583 scope.go:117] "RemoveContainer" containerID="bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"
Sep 08 13:48:59.085131 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:48:59.085110    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf\": container with ID starting with bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf not found: ID does not exist" containerID="bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"
Sep 08 13:48:59.085232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.085137    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf"} err="failed to get container status \"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf\": rpc error: code = NotFound desc = could not find container \"bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf\": container with ID starting with bf0c1a5f6fbcf6cf4fa55bfa50ca6edc843b7811b12732333a8439b62b70fccf not found: ID does not exist"
Sep 08 13:48:59.097873 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.097845    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:48:59.108133 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.108086    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-2d1742-6497d8f8d6-g9n9m"]
Sep 08 13:48:59.754795 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:48:59.754751    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:00.231869 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:00.231834    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" path="/var/lib/kubelet/pods/4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3/volumes"
Sep 08 13:49:00.942812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:00.942776    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:49:02.525084 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:02.525043    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:04.996472 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:04.996417    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:05.103962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.103927    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:05.283108 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.283077    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:49:05.283343 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.283318    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" containerID="cri-o://8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15" gracePeriod=30
Sep 08 13:49:05.312192 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.312153    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:05.416285 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.416236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:05.516025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.515989    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:05.760491 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:05.760389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:06.627401 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:06.627367    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:07.387258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:07.387202    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:10.237671 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.237632    2583 patch_prober.go:28] interesting pod/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 08 13:49:10.238044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.237687    2583 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 08 13:49:10.546925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.546899    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:49:10.606948 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.606906    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607115 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607013    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607115 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607037    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607115 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607082    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607113    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607143    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607169    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sksxd\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607252 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607227    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607267    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert\") pod \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\" (UID: \"5a2f44b2-fd79-4cfb-b592-8a31268c716b\") "
Sep 08 13:49:10.607450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607383    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:49:10.607450 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607394    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:49:10.607665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607444    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:49:10.607752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607730    2583 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-credential-socket\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.607963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607824    2583 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-socket\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.607963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607937    2583 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-data\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.607963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.607885    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:49:10.608257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.608080    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:10.609394 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.609372    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 08 13:49:10.609495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.609425    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:10.609541 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.609496    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd" (OuterVolumeSpecName: "kube-api-access-sksxd") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "kube-api-access-sksxd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:10.609603 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.609570    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a2f44b2-fd79-4cfb-b592-8a31268c716b" (UID: "5a2f44b2-fd79-4cfb-b592-8a31268c716b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:49:10.709251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709215    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sksxd\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-kube-api-access-sksxd\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.709251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709247    2583 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-workload-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.709251 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709258    2583 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istiod-ca-cert\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.709500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709266    2583 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-token\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.709500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709274    2583 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-podinfo\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:10.709500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:10.709282    2583 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a2f44b2-fd79-4cfb-b592-8a31268c716b-istio-envoy\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:11.150748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.150714    2583 generic.go:356] "Generic (PLEG): container finished" podID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerID="8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15" exitCode=0
Sep 08 13:49:11.150928 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.150759    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"
Sep 08 13:49:11.150928 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.150802    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerDied","Data":"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"}
Sep 08 13:49:11.150928 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.150864    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp" event={"ID":"5a2f44b2-fd79-4cfb-b592-8a31268c716b","Type":"ContainerDied","Data":"d72996657c839d72f23d205279711d963f6fd9d3f5cf9437d304e59cdcbf5295"}
Sep 08 13:49:11.150928 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.150893    2583 scope.go:117] "RemoveContainer" containerID="8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"
Sep 08 13:49:11.161208 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.161191    2583 scope.go:117] "RemoveContainer" containerID="5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"
Sep 08 13:49:11.170893 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.170799    2583 scope.go:117] "RemoveContainer" containerID="8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"
Sep 08 13:49:11.171577 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:11.171409    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15\": container with ID starting with 8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15 not found: ID does not exist" containerID="8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"
Sep 08 13:49:11.171577 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.171449    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15"} err="failed to get container status \"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15\": rpc error: code = NotFound desc = could not find container \"8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15\": container with ID starting with 8568ac6a671ddda99ec962408e154a7775b5990990c1d988c8b7b16d1ae13d15 not found: ID does not exist"
Sep 08 13:49:11.171577 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.171472    2583 scope.go:117] "RemoveContainer" containerID="5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"
Sep 08 13:49:11.184436 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:11.184398    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943\": container with ID starting with 5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943 not found: ID does not exist" containerID="5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"
Sep 08 13:49:11.184631 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.184573    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943"} err="failed to get container status \"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943\": rpc error: code = NotFound desc = could not find container \"5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943\": container with ID starting with 5bf554d93fc67b884a57a406b6709747a0fe24deb9e3ac1b1536af71469d3943 not found: ID does not exist"
Sep 08 13:49:11.186693 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.186674    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:49:11.189582 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:11.189545    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-2d1742-openshift-default-57856c7dcd-bczfp"]
Sep 08 13:49:12.232037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:12.232003    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" path="/var/lib/kubelet/pods/5a2f44b2-fd79-4cfb-b592-8a31268c716b/volumes"
Sep 08 13:49:16.093212 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.093172    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:49:16.093651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.093463    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" podUID="468d74a0-bbf3-42b9-972e-4eda51ad3501" containerName="maas-api" containerID="cri-o://af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042" gracePeriod=30
Sep 08 13:49:16.122965 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.122930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:16.254170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.254136    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:49:16.254477 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.254402    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" podUID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" containerName="maas-api" containerID="cri-o://b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e" gracePeriod=30
Sep 08 13:49:16.395427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.395398    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:49:16.472111 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.472082    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") pod \"468d74a0-bbf3-42b9-972e-4eda51ad3501\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") "
Sep 08 13:49:16.472299 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.472151    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npldg\" (UniqueName: \"kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg\") pod \"468d74a0-bbf3-42b9-972e-4eda51ad3501\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") "
Sep 08 13:49:16.472299 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.472238    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca\") pod \"468d74a0-bbf3-42b9-972e-4eda51ad3501\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") "
Sep 08 13:49:16.472299 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.472295    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs\") pod \"468d74a0-bbf3-42b9-972e-4eda51ad3501\" (UID: \"468d74a0-bbf3-42b9-972e-4eda51ad3501\") "
Sep 08 13:49:16.472787 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.472750    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "468d74a0-bbf3-42b9-972e-4eda51ad3501" (UID: "468d74a0-bbf3-42b9-972e-4eda51ad3501"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:16.475726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.475616    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "468d74a0-bbf3-42b9-972e-4eda51ad3501" (UID: "468d74a0-bbf3-42b9-972e-4eda51ad3501"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:49:16.475847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.475728    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "468d74a0-bbf3-42b9-972e-4eda51ad3501" (UID: "468d74a0-bbf3-42b9-972e-4eda51ad3501"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:49:16.475991 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.475944    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg" (OuterVolumeSpecName: "kube-api-access-npldg") pod "468d74a0-bbf3-42b9-972e-4eda51ad3501" (UID: "468d74a0-bbf3-42b9-972e-4eda51ad3501"). InnerVolumeSpecName "kube-api-access-npldg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:16.507604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.507575    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:49:16.541842 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.541795    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:16.573363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573327    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") pod \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") "
Sep 08 13:49:16.573571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573395    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72d66\" (UniqueName: \"kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66\") pod \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") "
Sep 08 13:49:16.573571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573440    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca\") pod \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") "
Sep 08 13:49:16.573571 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573500    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs\") pod \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\" (UID: \"1cee1059-f5bd-484d-a5c5-79a0f542a9d6\") "
Sep 08 13:49:16.573857 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573836    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.573920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573860    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/468d74a0-bbf3-42b9-972e-4eda51ad3501-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.573920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573876    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npldg\" (UniqueName: \"kubernetes.io/projected/468d74a0-bbf3-42b9-972e-4eda51ad3501-kube-api-access-npldg\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.573920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573887    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/468d74a0-bbf3-42b9-972e-4eda51ad3501-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.573920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.573875    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1cee1059-f5bd-484d-a5c5-79a0f542a9d6" (UID: "1cee1059-f5bd-484d-a5c5-79a0f542a9d6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:49:16.575870 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.575833    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1cee1059-f5bd-484d-a5c5-79a0f542a9d6" (UID: "1cee1059-f5bd-484d-a5c5-79a0f542a9d6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:49:16.576000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.575947    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1cee1059-f5bd-484d-a5c5-79a0f542a9d6" (UID: "1cee1059-f5bd-484d-a5c5-79a0f542a9d6"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:49:16.576000 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.575963    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66" (OuterVolumeSpecName: "kube-api-access-72d66") pod "1cee1059-f5bd-484d-a5c5-79a0f542a9d6" (UID: "1cee1059-f5bd-484d-a5c5-79a0f542a9d6"). InnerVolumeSpecName "kube-api-access-72d66". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:49:16.674951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.674865    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72d66\" (UniqueName: \"kubernetes.io/projected/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-kube-api-access-72d66\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.674951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.674898    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.674951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.674909    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:16.674951 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:16.674918    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1cee1059-f5bd-484d-a5c5-79a0f542a9d6-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:49:17.016878 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.016789    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:17.110708 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.110675    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:17.192308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.192272    2583 generic.go:356] "Generic (PLEG): container finished" podID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" containerID="b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e" exitCode=0
Sep 08 13:49:17.192490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.192346    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"
Sep 08 13:49:17.192490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.192361    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" event={"ID":"1cee1059-f5bd-484d-a5c5-79a0f542a9d6","Type":"ContainerDied","Data":"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"}
Sep 08 13:49:17.192490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.192436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5" event={"ID":"1cee1059-f5bd-484d-a5c5-79a0f542a9d6","Type":"ContainerDied","Data":"f5470c74276a9bd68feb816fd4baec2dd630a7b7349c93234bb3932521a8c6ab"}
Sep 08 13:49:17.192490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.192458    2583 scope.go:117] "RemoveContainer" containerID="b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"
Sep 08 13:49:17.194034 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.194011    2583 generic.go:356] "Generic (PLEG): container finished" podID="468d74a0-bbf3-42b9-972e-4eda51ad3501" containerID="af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042" exitCode=0
Sep 08 13:49:17.194109 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.194072    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"
Sep 08 13:49:17.194161 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.194116    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" event={"ID":"468d74a0-bbf3-42b9-972e-4eda51ad3501","Type":"ContainerDied","Data":"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"}
Sep 08 13:49:17.194202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.194170    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6" event={"ID":"468d74a0-bbf3-42b9-972e-4eda51ad3501","Type":"ContainerDied","Data":"485d29868b0d98559cb9a263eae8277ae52c7922e184ec09abe92beed54d95db"}
Sep 08 13:49:17.204409 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.204389    2583 scope.go:117] "RemoveContainer" containerID="b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"
Sep 08 13:49:17.204859 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:17.204836    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e\": container with ID starting with b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e not found: ID does not exist" containerID="b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"
Sep 08 13:49:17.204929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.204872    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e"} err="failed to get container status \"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e\": rpc error: code = NotFound desc = could not find container \"b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e\": container with ID starting with b84aafae0e9c71fc30d1aed91a33dfaad65928f8a8117b2dc599a83ac499954e not found: ID does not exist"
Sep 08 13:49:17.204929 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.204896    2583 scope.go:117] "RemoveContainer" containerID="af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"
Sep 08 13:49:17.213603 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.213582    2583 scope.go:117] "RemoveContainer" containerID="af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"
Sep 08 13:49:17.213859 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:17.213838    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042\": container with ID starting with af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042 not found: ID does not exist" containerID="af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"
Sep 08 13:49:17.213931 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.213870    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042"} err="failed to get container status \"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042\": rpc error: code = NotFound desc = could not find container \"af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042\": container with ID starting with af5aada78c76524511c54a29a0495b5a8200ee98f2b91136eba590baea225042 not found: ID does not exist"
Sep 08 13:49:17.220106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.220084    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:49:17.226422 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.226398    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-f8154283-9d69b944c-4ddx6"]
Sep 08 13:49:17.237188 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.237163    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:49:17.240898 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:17.240868    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-a7ef4b-746b9d88f6-r47g5"]
Sep 08 13:49:18.231137 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:18.231104    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" path="/var/lib/kubelet/pods/1cee1059-f5bd-484d-a5c5-79a0f542a9d6/volumes"
Sep 08 13:49:18.233624 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:18.231482    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="468d74a0-bbf3-42b9-972e-4eda51ad3501" path="/var/lib/kubelet/pods/468d74a0-bbf3-42b9-972e-4eda51ad3501/volumes"
Sep 08 13:49:18.676988 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:18.676947    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:49:23.169673 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:23.169637    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:23.261462 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:23.261421    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:23.498352 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:23.498268    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:23.741210 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:23.741168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:24.821514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:24.821480    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:24.922909 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:24.922873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:25.068813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:25.068769    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:25.232509 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:25.232428    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:26.139178 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:26.139140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:49:28.710712 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:28.710677    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:28.927099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:28.927061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:29.825074 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.825035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:29.960855 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.960818    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 13:49:29.961470 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961455    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" containerName="maas-api"
Sep 08 13:49:29.961519 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961475    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" containerName="maas-api"
Sep 08 13:49:29.961519 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961502    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:29.961519 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961510    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:29.961641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961526    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" containerName="maas-api"
Sep 08 13:49:29.961641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961535    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" containerName="maas-api"
Sep 08 13:49:29.961641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961564    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="468d74a0-bbf3-42b9-972e-4eda51ad3501" containerName="maas-api"
Sep 08 13:49:29.961641 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961573    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="468d74a0-bbf3-42b9-972e-4eda51ad3501" containerName="maas-api"
Sep 08 13:49:29.961768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961672    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="4cc0ee64-c6bd-44c9-a2dd-91231f9c4bb3" containerName="maas-api"
Sep 08 13:49:29.961768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961685    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:29.961768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961695    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:29.961768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961706    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="1cee1059-f5bd-484d-a5c5-79a0f542a9d6" containerName="maas-api"
Sep 08 13:49:29.961768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.961717    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="468d74a0-bbf3-42b9-972e-4eda51ad3501" containerName="maas-api"
Sep 08 13:49:29.966761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.966738    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:29.969013 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.968989    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-1f082e-serving-cert\""
Sep 08 13:49:29.974735 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.974710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 13:49:29.999717 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:29.999685    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:30.101170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.101070    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:30.110301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.110272    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt5jl\" (UniqueName: \"kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.110439 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.110323    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.110439 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.110373    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.110570 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.110463    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.212349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.212305    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.212578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.212481    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt5jl\" (UniqueName: \"kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.212578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.212523    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.212578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.212544    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.212799 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:30.212703    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-serving-cert: secret "maas-api-e2e-shared-a-w3-1f082e-serving-cert" not found
Sep 08 13:49:30.212799 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:30.212782    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls podName:a1d828dc-be7c-463d-b488-42650a5d5b51 nodeName:}" failed. No retries permitted until 2026-09-08 13:49:30.712750999 +0000 UTC m=+1256.930667325 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls") pod "maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" (UID: "a1d828dc-be7c-463d-b488-42650a5d5b51") : secret "maas-api-e2e-shared-a-w3-1f082e-serving-cert" not found
Sep 08 13:49:30.213866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.213839    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.215448 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.215425    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.224070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.224046    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt5jl\" (UniqueName: \"kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.536594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.536533    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:30.636228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.636194    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:30.718539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.718496    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.721184 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.721147    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:30.770116 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.770074    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:30.881615 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:30.881502    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:31.072406 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.072375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:31.157057 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.157029    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 13:49:31.159597 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:49:31.159532    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d828dc_be7c_463d_b488_42650a5d5b51.slice/crio-ae38b3ef1be078866cd6913efe9bf55ea0259130a4a0376baa5483e4bc0e8661 WatchSource:0}: Error finding container ae38b3ef1be078866cd6913efe9bf55ea0259130a4a0376baa5483e4bc0e8661: Status 404 returned error can't find the container with id ae38b3ef1be078866cd6913efe9bf55ea0259130a4a0376baa5483e4bc0e8661
Sep 08 13:49:31.240410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.240379    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:31.283047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.283017    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" event={"ID":"a1d828dc-be7c-463d-b488-42650a5d5b51","Type":"ContainerStarted","Data":"ae38b3ef1be078866cd6913efe9bf55ea0259130a4a0376baa5483e4bc0e8661"}
Sep 08 13:49:31.327404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.327370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:31.480758 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:31.480674    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:33.296973 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:33.296933    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" event={"ID":"a1d828dc-be7c-463d-b488-42650a5d5b51","Type":"ContainerStarted","Data":"e8e35792e26b2daed1174e75cb94f592d011660ef5304dd8f62ed9e490d51af1"}
Sep 08 13:49:33.297423 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:33.297085    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:33.326198 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:33.326137    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" podStartSLOduration=3.07667145 podStartE2EDuration="4.326118996s" totalImagesPullingTime="1.249447546s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:49:33.324052811 +0000 UTC m=+1259.541969135" watchObservedRunningTime="2026-09-08 13:49:33.326118996 +0000 UTC m=+1259.544035321"
Sep 08 13:49:33.557516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:33.557426    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:49:37.175324 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:37.175284    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:37.261852 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:37.261818    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:38.169511 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:38.169482    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:38.538037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:38.537997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:39.258121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:39.258064    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:39.309431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:39.309402    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 13:49:42.110041 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:42.110009    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:43.156726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.156683    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:43.439494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.439398    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:49:43.439941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.439925    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:43.439941 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.439943    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a2f44b2-fd79-4cfb-b592-8a31268c716b" containerName="istio-proxy"
Sep 08 13:49:43.445061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.445036    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.447632 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.447608    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-3c9fc8-serving-cert\""
Sep 08 13:49:43.457795 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.457681    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:49:43.491626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.489542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:43.539649 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.539611    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.539847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.539670    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.539847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.539763    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hh45m\" (UniqueName: \"kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.539956 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.539850    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.587305 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.587258    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:43.640765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.640729    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.640930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.640789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.640930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.640827    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hh45m\" (UniqueName: \"kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.640930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.640889    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.641703 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.641476    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.643500 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.643476    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.643614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.643489    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.649718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.649696    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hh45m\" (UniqueName: \"kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m\") pod \"maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:43.649970 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.649944    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:43.759675 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:43.759594    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:44.026250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:44.026207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:44.072315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:44.072278    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:49:44.073619 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:49:44.073578    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35d35d85_9bfc_4017_8c10_f484ebf9c401.slice/crio-ff38aa17ed9bbdec3e3e544c1d37a7042f9c4b7bf73b413f365ca34120a82207 WatchSource:0}: Error finding container ff38aa17ed9bbdec3e3e544c1d37a7042f9c4b7bf73b413f365ca34120a82207: Status 404 returned error can't find the container with id ff38aa17ed9bbdec3e3e544c1d37a7042f9c4b7bf73b413f365ca34120a82207
Sep 08 13:49:44.116025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:44.115990    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:44.372359 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:44.372212    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" event={"ID":"35d35d85-9bfc-4017-8c10-f484ebf9c401","Type":"ContainerStarted","Data":"ff38aa17ed9bbdec3e3e544c1d37a7042f9c4b7bf73b413f365ca34120a82207"}
Sep 08 13:49:45.381604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:45.381541    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" event={"ID":"35d35d85-9bfc-4017-8c10-f484ebf9c401","Type":"ContainerStarted","Data":"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"}
Sep 08 13:49:45.382038 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:45.381746    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:45.404636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:45.403278    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" podStartSLOduration=1.645082621 podStartE2EDuration="2.403255842s" totalImagesPullingTime="758.173221ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:49:45.402796112 +0000 UTC m=+1271.620712450" watchObservedRunningTime="2026-09-08 13:49:45.403255842 +0000 UTC m=+1271.621172168"
Sep 08 13:49:47.454612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:47.454565    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:49:51.083258 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:51.083225    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:51.393315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:51.393232    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:49:53.694635 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:53.694600    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:54.026776 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:54.026738    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:54.345891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:54.345802    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:54.526307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:54.526276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:57.810407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:57.810375    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:57.901622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:57.901584    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.065404 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.065368    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 13:49:59.070896 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.070838    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.084821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.084789    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-905fec-serving-cert\""
Sep 08 13:49:59.091111 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.091077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 13:49:59.103104 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.103073    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.202900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.202859    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.203086 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.202920    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps2fk\" (UniqueName: \"kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.203135 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.203089    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.203172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.203135    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.218292 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.218260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.304174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.304134    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.304174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.304184    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ps2fk\" (UniqueName: \"kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.304443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.304311    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.304443 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.304351    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.304443 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:59.304427    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-serving-cert: secret "maas-api-e2e-shared-b-w3-905fec-serving-cert" not found
Sep 08 13:49:59.304654 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:49:59.304519    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls podName:b5d3838f-671a-4eeb-a6ee-8c53744ae353 nodeName:}" failed. No retries permitted until 2026-09-08 13:49:59.804496595 +0000 UTC m=+1286.022412899 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls") pod "maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" (UID: "b5d3838f-671a-4eeb-a6ee-8c53744ae353") : secret "maas-api-e2e-shared-b-w3-905fec-serving-cert" not found
Sep 08 13:49:59.305166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.305124    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.307039 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.307018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.312665 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.312642    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps2fk\" (UniqueName: \"kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.468669 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.468574    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.678651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.678542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.783137 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.783099    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:49:59.809343 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.809309    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.812028 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.811993    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:49:59.999389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:49:59.999352    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:50:00.273429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:00.273406    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 13:50:00.274751 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:00.274726    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d3838f_671a_4eeb_a6ee_8c53744ae353.slice/crio-481dc998b0811aa2f3fe9bcdc8e5e8200f1a4b84b9c7ebec0694e1a2b14ffa48 WatchSource:0}: Error finding container 481dc998b0811aa2f3fe9bcdc8e5e8200f1a4b84b9c7ebec0694e1a2b14ffa48: Status 404 returned error can't find the container with id 481dc998b0811aa2f3fe9bcdc8e5e8200f1a4b84b9c7ebec0694e1a2b14ffa48
Sep 08 13:50:00.484177 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:00.484098    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" event={"ID":"b5d3838f-671a-4eeb-a6ee-8c53744ae353","Type":"ContainerStarted","Data":"481dc998b0811aa2f3fe9bcdc8e5e8200f1a4b84b9c7ebec0694e1a2b14ffa48"}
Sep 08 13:50:01.492239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:01.492197    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" event={"ID":"b5d3838f-671a-4eeb-a6ee-8c53744ae353","Type":"ContainerStarted","Data":"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"}
Sep 08 13:50:01.492768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:01.492323    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:50:01.511165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:01.511103    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" podStartSLOduration=2.176946866 podStartE2EDuration="2.511086374s" totalImagesPullingTime="334.139508ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:49:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:01.507889161 +0000 UTC m=+1287.725805498" watchObservedRunningTime="2026-09-08 13:50:01.511086374 +0000 UTC m=+1287.729002746"
Sep 08 13:50:04.455185 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:04.455140    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:50:06.060426 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:06.060393    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:07.021041 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:07.020995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:07.367056 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:07.366968    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:07.503971 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:07.503944    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 13:50:07.694493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:07.694406    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:08.432437 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:08.432366    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:10.817203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:10.817165    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:10.916604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:10.916543    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:11.938753 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:11.938722    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:50:11.943101 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:11.943079    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:11.945525 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:11.945500    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-d9970e-serving-cert\""
Sep 08 13:50:11.952021 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:11.951994    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:50:11.966786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:11.966744    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:12.027388 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.027353    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:12.035580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.035527    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.035718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.035650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88t59\" (UniqueName: \"kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.035787 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.035762    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.035851 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.035806    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.137031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.136992    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.137031 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.137048    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.137283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.137086    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.137283 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:50:12.137168    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-serving-cert: secret "maas-api-e2e-shared-b-w2-d9970e-serving-cert" not found
Sep 08 13:50:12.137283 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:50:12.137245    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls podName:350be2e4-e363-46a2-b616-b1e92a1c3188 nodeName:}" failed. No retries permitted until 2026-09-08 13:50:12.637227792 +0000 UTC m=+1298.855144099 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls") pod "maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" (UID: "350be2e4-e363-46a2-b616-b1e92a1c3188") : secret "maas-api-e2e-shared-b-w2-d9970e-serving-cert" not found
Sep 08 13:50:12.137283 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.137270    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88t59\" (UniqueName: \"kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.137945 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.137869    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.139778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.139755    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.148660 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.148632    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88t59\" (UniqueName: \"kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.408433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.408399    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:12.586428 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.586395    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:12.643123 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.643083    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.645727 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.645703    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:12.683334 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.683239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:12.858040 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:12.857999    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:13.137219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:13.137191    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:50:13.139749 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:13.139726    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod350be2e4_e363_46a2_b616_b1e92a1c3188.slice/crio-2d3e2caedc8a91b18ff995a5fda9d5b07961a9cdae4c54b65cf3c5c1a3d59771 WatchSource:0}: Error finding container 2d3e2caedc8a91b18ff995a5fda9d5b07961a9cdae4c54b65cf3c5c1a3d59771: Status 404 returned error can't find the container with id 2d3e2caedc8a91b18ff995a5fda9d5b07961a9cdae4c54b65cf3c5c1a3d59771
Sep 08 13:50:13.567473 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:13.567436    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" event={"ID":"350be2e4-e363-46a2-b616-b1e92a1c3188","Type":"ContainerStarted","Data":"2d3e2caedc8a91b18ff995a5fda9d5b07961a9cdae4c54b65cf3c5c1a3d59771"}
Sep 08 13:50:15.584224 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:15.584179    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" event={"ID":"350be2e4-e363-46a2-b616-b1e92a1c3188","Type":"ContainerStarted","Data":"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"}
Sep 08 13:50:15.584700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:15.584322    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:15.605200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:15.605136    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" podStartSLOduration=2.725594162 podStartE2EDuration="4.605120151s" totalImagesPullingTime="1.879525989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:15.602339866 +0000 UTC m=+1301.820256202" watchObservedRunningTime="2026-09-08 13:50:15.605120151 +0000 UTC m=+1301.823036486"
Sep 08 13:50:17.244055 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:17.244017    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:50:19.629106 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:19.629050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:21.596577 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:21.596519    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:50:21.628110 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:21.628077    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:22.749498 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:22.749455    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:33.049863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:33.049828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:35.854312 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:35.854269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:36.543193 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.543154    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:36.547259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.547232    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:36.549534 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.549384    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:36.549724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.549702    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qzs56\""
Sep 08 13:50:36.552385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.552361    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:36.686640 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.686604    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhx9q\" (UniqueName: \"kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q\") pod \"test-curl-7717-140358673515984\" (UID: \"58d72fac-39ff-4733-9a4f-e7c04f172764\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:36.788280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.788248    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhx9q\" (UniqueName: \"kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q\") pod \"test-curl-7717-140358673515984\" (UID: \"58d72fac-39ff-4733-9a4f-e7c04f172764\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:36.796356 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.796287    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhx9q\" (UniqueName: \"kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q\") pod \"test-curl-7717-140358673515984\" (UID: \"58d72fac-39ff-4733-9a4f-e7c04f172764\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:36.858517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:36.858475    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:37.138610 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:37.138580    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:37.140001 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:37.139978    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58d72fac_39ff_4733_9a4f_e7c04f172764.slice/crio-030c434a062ce5100f8a273b6a4695adb62018957e0a992585ba461e757226b3 WatchSource:0}: Error finding container 030c434a062ce5100f8a273b6a4695adb62018957e0a992585ba461e757226b3: Status 404 returned error can't find the container with id 030c434a062ce5100f8a273b6a4695adb62018957e0a992585ba461e757226b3
Sep 08 13:50:37.141906 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:37.141889    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:50:37.740391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:37.740361    2583 generic.go:356] "Generic (PLEG): container finished" podID="58d72fac-39ff-4733-9a4f-e7c04f172764" containerID="83fbf06640cdc896e479649d54de36bc23f75b48d58384b532f0159a5430f4f1" exitCode=0
Sep 08 13:50:37.740580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:37.740453    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673515984" event={"ID":"58d72fac-39ff-4733-9a4f-e7c04f172764","Type":"ContainerDied","Data":"83fbf06640cdc896e479649d54de36bc23f75b48d58384b532f0159a5430f4f1"}
Sep 08 13:50:37.740580 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:37.740499    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673515984" event={"ID":"58d72fac-39ff-4733-9a4f-e7c04f172764","Type":"ContainerStarted","Data":"030c434a062ce5100f8a273b6a4695adb62018957e0a992585ba461e757226b3"}
Sep 08 13:50:38.898486 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:38.898460    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:38.918821 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:38.918798    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140358673515984_58d72fac-39ff-4733-9a4f-e7c04f172764/test-curl-7717-140358673515984/0.log"
Sep 08 13:50:38.942502 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:38.942466    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:38.946076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:38.946051    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:39.011474 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.011364    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhx9q\" (UniqueName: \"kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q\") pod \"58d72fac-39ff-4733-9a4f-e7c04f172764\" (UID: \"58d72fac-39ff-4733-9a4f-e7c04f172764\") "
Sep 08 13:50:39.013784 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.013750    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q" (OuterVolumeSpecName: "kube-api-access-jhx9q") pod "58d72fac-39ff-4733-9a4f-e7c04f172764" (UID: "58d72fac-39ff-4733-9a4f-e7c04f172764"). InnerVolumeSpecName "kube-api-access-jhx9q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:50:39.065835 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.065796    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:39.066295 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.066280    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58d72fac-39ff-4733-9a4f-e7c04f172764" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:39.066345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.066298    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="58d72fac-39ff-4733-9a4f-e7c04f172764" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:39.066385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.066378    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="58d72fac-39ff-4733-9a4f-e7c04f172764" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:39.069366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.069349    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.074349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.074317    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:39.112419 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.112385    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhx9q\" (UniqueName: \"kubernetes.io/projected/58d72fac-39ff-4733-9a4f-e7c04f172764-kube-api-access-jhx9q\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:50:39.213345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.213300    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8dpv\" (UniqueName: \"kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv\") pod \"test-curl-7717-140358673515984\" (UID: \"08f0694c-583f-46af-9644-0ad11b61a309\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.246499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.246454    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:50:39.314058 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.314023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m8dpv\" (UniqueName: \"kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv\") pod \"test-curl-7717-140358673515984\" (UID: \"08f0694c-583f-46af-9644-0ad11b61a309\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.321656 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.321625    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8dpv\" (UniqueName: \"kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv\") pod \"test-curl-7717-140358673515984\" (UID: \"08f0694c-583f-46af-9644-0ad11b61a309\") " pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.381307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.381264    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.659222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.659190    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:39.660369 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:39.660346    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod08f0694c_583f_46af_9644_0ad11b61a309.slice/crio-a5aa66d14b45e7b26ada0ea3e953d4ea39999422b5c65af65926bd48bcdb93fa WatchSource:0}: Error finding container a5aa66d14b45e7b26ada0ea3e953d4ea39999422b5c65af65926bd48bcdb93fa: Status 404 returned error can't find the container with id a5aa66d14b45e7b26ada0ea3e953d4ea39999422b5c65af65926bd48bcdb93fa
Sep 08 13:50:39.755620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.755594    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:39.755620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.755621    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="030c434a062ce5100f8a273b6a4695adb62018957e0a992585ba461e757226b3"
Sep 08 13:50:39.756990 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.756962    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673515984" event={"ID":"08f0694c-583f-46af-9644-0ad11b61a309","Type":"ContainerStarted","Data":"a5aa66d14b45e7b26ada0ea3e953d4ea39999422b5c65af65926bd48bcdb93fa"}
Sep 08 13:50:39.769486 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:39.769455    2583 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7717-140358673515984" oldPodUID="58d72fac-39ff-4733-9a4f-e7c04f172764" podUID="08f0694c-583f-46af-9644-0ad11b61a309"
Sep 08 13:50:40.236385 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:40.236347    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58d72fac-39ff-4733-9a4f-e7c04f172764" path="/var/lib/kubelet/pods/58d72fac-39ff-4733-9a4f-e7c04f172764/volumes"
Sep 08 13:50:40.765250 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:40.765214    2583 generic.go:356] "Generic (PLEG): container finished" podID="08f0694c-583f-46af-9644-0ad11b61a309" containerID="7c728497303baf9a1012b80648484d1e622599dca488c08cd26a7b5edb3723c8" exitCode=0
Sep 08 13:50:40.765427 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:40.765305    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673515984" event={"ID":"08f0694c-583f-46af-9644-0ad11b61a309","Type":"ContainerDied","Data":"7c728497303baf9a1012b80648484d1e622599dca488c08cd26a7b5edb3723c8"}
Sep 08 13:50:41.917818 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:41.917791    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:41.935911 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:41.935865    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140358673515984_08f0694c-583f-46af-9644-0ad11b61a309/test-curl-7717-140358673515984/0.log"
Sep 08 13:50:41.959345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:41.959305    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:41.963059 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:41.963030    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140358673515984"]
Sep 08 13:50:42.039884 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.039794    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8dpv\" (UniqueName: \"kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv\") pod \"08f0694c-583f-46af-9644-0ad11b61a309\" (UID: \"08f0694c-583f-46af-9644-0ad11b61a309\") "
Sep 08 13:50:42.042095 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.042068    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv" (OuterVolumeSpecName: "kube-api-access-m8dpv") pod "08f0694c-583f-46af-9644-0ad11b61a309" (UID: "08f0694c-583f-46af-9644-0ad11b61a309"). InnerVolumeSpecName "kube-api-access-m8dpv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:50:42.092357 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.092321    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:42.092795 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.092782    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08f0694c-583f-46af-9644-0ad11b61a309" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:42.092855 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.092796    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="08f0694c-583f-46af-9644-0ad11b61a309" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:42.092890 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.092885    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="08f0694c-583f-46af-9644-0ad11b61a309" containerName="test-curl-7717-140358673515984"
Sep 08 13:50:42.096022 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.096007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:42.100314 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.100286    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:42.140963 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.140927    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8dpv\" (UniqueName: \"kubernetes.io/projected/08f0694c-583f-46af-9644-0ad11b61a309-kube-api-access-m8dpv\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:50:42.231854 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.231820    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="08f0694c-583f-46af-9644-0ad11b61a309" path="/var/lib/kubelet/pods/08f0694c-583f-46af-9644-0ad11b61a309/volumes"
Sep 08 13:50:42.241720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.241698    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5bhp\" (UniqueName: \"kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp\") pod \"test-curl-7717-140358673516560\" (UID: \"7ddd5d0f-70fa-4eb4-8327-348eefd3030d\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:42.343167 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.343067    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5bhp\" (UniqueName: \"kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp\") pod \"test-curl-7717-140358673516560\" (UID: \"7ddd5d0f-70fa-4eb4-8327-348eefd3030d\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:42.351757 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.351728    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5bhp\" (UniqueName: \"kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp\") pod \"test-curl-7717-140358673516560\" (UID: \"7ddd5d0f-70fa-4eb4-8327-348eefd3030d\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:42.408175 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.408140    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:42.684387 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.684360    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:42.685650 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:42.685626    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7ddd5d0f_70fa_4eb4_8327_348eefd3030d.slice/crio-c20519a032cafce26faf86da8eb5d9eda6e4776dcc77348918250283c2ee11d4 WatchSource:0}: Error finding container c20519a032cafce26faf86da8eb5d9eda6e4776dcc77348918250283c2ee11d4: Status 404 returned error can't find the container with id c20519a032cafce26faf86da8eb5d9eda6e4776dcc77348918250283c2ee11d4
Sep 08 13:50:42.779872 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.779839    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673515984"
Sep 08 13:50:42.780074 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.779906    2583 scope.go:117] "RemoveContainer" containerID="7c728497303baf9a1012b80648484d1e622599dca488c08cd26a7b5edb3723c8"
Sep 08 13:50:42.781837 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:42.781798    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673516560" event={"ID":"7ddd5d0f-70fa-4eb4-8327-348eefd3030d","Type":"ContainerStarted","Data":"c20519a032cafce26faf86da8eb5d9eda6e4776dcc77348918250283c2ee11d4"}
Sep 08 13:50:43.793279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:43.793236    2583 generic.go:356] "Generic (PLEG): container finished" podID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" containerID="e8555730cdf7d94c45806e1878ab8f2d6e9b04a0eef5094f28460907b603ead1" exitCode=0
Sep 08 13:50:43.793799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:43.793326    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673516560" event={"ID":"7ddd5d0f-70fa-4eb4-8327-348eefd3030d","Type":"ContainerDied","Data":"e8555730cdf7d94c45806e1878ab8f2d6e9b04a0eef5094f28460907b603ead1"}
Sep 08 13:50:44.937391 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.937362    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:44.957502 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.957474    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140358673516560_7ddd5d0f-70fa-4eb4-8327-348eefd3030d/test-curl-7717-140358673516560/0.log"
Sep 08 13:50:44.969164 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.969123    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5bhp\" (UniqueName: \"kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp\") pod \"7ddd5d0f-70fa-4eb4-8327-348eefd3030d\" (UID: \"7ddd5d0f-70fa-4eb4-8327-348eefd3030d\") "
Sep 08 13:50:44.971290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.971263    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp" (OuterVolumeSpecName: "kube-api-access-g5bhp") pod "7ddd5d0f-70fa-4eb4-8327-348eefd3030d" (UID: "7ddd5d0f-70fa-4eb4-8327-348eefd3030d"). InnerVolumeSpecName "kube-api-access-g5bhp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:50:44.982605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.982536    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:44.985613 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:44.985589    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:45.070488 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.070389    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5bhp\" (UniqueName: \"kubernetes.io/projected/7ddd5d0f-70fa-4eb4-8327-348eefd3030d-kube-api-access-g5bhp\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:50:45.123856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.123822    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:45.124288 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.124275    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:45.124333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.124291    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:45.124368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.124360    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:45.127614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.127598    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.135036 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.135006    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:45.171189 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.171150    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzhq7\" (UniqueName: \"kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7\") pod \"test-curl-7717-140358673516560\" (UID: \"a97ac188-6f19-47b4-a702-7288ec30010b\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.272650 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.272611    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzhq7\" (UniqueName: \"kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7\") pod \"test-curl-7717-140358673516560\" (UID: \"a97ac188-6f19-47b4-a702-7288ec30010b\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.281670 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.281629    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzhq7\" (UniqueName: \"kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7\") pod \"test-curl-7717-140358673516560\" (UID: \"a97ac188-6f19-47b4-a702-7288ec30010b\") " pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.438803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.438712    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.712465 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.712436    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:45.713861 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:45.713831    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda97ac188_6f19_47b4_a702_7288ec30010b.slice/crio-2fc1e2e76fb3e61debe04e53e142c7d41e10f0c7edd9856c98aa45f1cc25bad5 WatchSource:0}: Error finding container 2fc1e2e76fb3e61debe04e53e142c7d41e10f0c7edd9856c98aa45f1cc25bad5: Status 404 returned error can't find the container with id 2fc1e2e76fb3e61debe04e53e142c7d41e10f0c7edd9856c98aa45f1cc25bad5
Sep 08 13:50:45.807825 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.807791    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673516560" event={"ID":"a97ac188-6f19-47b4-a702-7288ec30010b","Type":"ContainerStarted","Data":"2fc1e2e76fb3e61debe04e53e142c7d41e10f0c7edd9856c98aa45f1cc25bad5"}
Sep 08 13:50:45.809132 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.809110    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c20519a032cafce26faf86da8eb5d9eda6e4776dcc77348918250283c2ee11d4"
Sep 08 13:50:45.809263 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.809139    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:45.820639 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:45.820613    2583 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7717-140358673516560" oldPodUID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" podUID="a97ac188-6f19-47b4-a702-7288ec30010b"
Sep 08 13:50:46.232788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:46.232755    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7ddd5d0f-70fa-4eb4-8327-348eefd3030d" path="/var/lib/kubelet/pods/7ddd5d0f-70fa-4eb4-8327-348eefd3030d/volumes"
Sep 08 13:50:46.818254 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:46.818211    2583 generic.go:356] "Generic (PLEG): container finished" podID="a97ac188-6f19-47b4-a702-7288ec30010b" containerID="de81f3ee2128fc1c169fff946f4256c2bec10317cba193e581dfa35dee46c227" exitCode=0
Sep 08 13:50:46.818436 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:46.818264    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7717-140358673516560" event={"ID":"a97ac188-6f19-47b4-a702-7288ec30010b","Type":"ContainerDied","Data":"de81f3ee2128fc1c169fff946f4256c2bec10317cba193e581dfa35dee46c227"}
Sep 08 13:50:47.956327 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:47.956300    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:47.975925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:47.975844    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7717-140358673516560_a97ac188-6f19-47b4-a702-7288ec30010b/test-curl-7717-140358673516560/0.log"
Sep 08 13:50:47.998871 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:47.998838    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzhq7\" (UniqueName: \"kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7\") pod \"a97ac188-6f19-47b4-a702-7288ec30010b\" (UID: \"a97ac188-6f19-47b4-a702-7288ec30010b\") "
Sep 08 13:50:48.001771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.001738    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7" (OuterVolumeSpecName: "kube-api-access-tzhq7") pod "a97ac188-6f19-47b4-a702-7288ec30010b" (UID: "a97ac188-6f19-47b4-a702-7288ec30010b"). InnerVolumeSpecName "kube-api-access-tzhq7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:50:48.002596 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.002545    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:48.005964 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.005939    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7717-140358673516560"]
Sep 08 13:50:48.100507 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.100427    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzhq7\" (UniqueName: \"kubernetes.io/projected/a97ac188-6f19-47b4-a702-7288ec30010b-kube-api-access-tzhq7\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:50:48.232453 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.232417    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a97ac188-6f19-47b4-a702-7288ec30010b" path="/var/lib/kubelet/pods/a97ac188-6f19-47b4-a702-7288ec30010b/volumes"
Sep 08 13:50:48.610917 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.610884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:48.832579 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.832523    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7717-140358673516560"
Sep 08 13:50:48.832781 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.832591    2583 scope.go:117] "RemoveContainer" containerID="de81f3ee2128fc1c169fff946f4256c2bec10317cba193e581dfa35dee46c227"
Sep 08 13:50:48.944514 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:48.944424    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:49.053165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:49.053113    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:52.439429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:52.439397    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:52.542270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:52.542236    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:53.751981 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.751928    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:50:53.753349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.753317    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a97ac188-6f19-47b4-a702-7288ec30010b" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:53.753349 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.753344    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a97ac188-6f19-47b4-a702-7288ec30010b" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:53.753536 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.753505    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a97ac188-6f19-47b4-a702-7288ec30010b" containerName="test-curl-7717-140358673516560"
Sep 08 13:50:53.758144 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.758120    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.760403 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.760377    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-2ee58a5f-serving-cert\""
Sep 08 13:50:53.770428 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.770400    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:50:53.782812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.782774    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:53.860203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.860172    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.860203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.860207    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.860463 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.860273    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mzg7\" (UniqueName: \"kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.860463 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.860314    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.892836 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.892801    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:53.961637 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.961603    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.961637 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.961641    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.961901 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.961683    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mzg7\" (UniqueName: \"kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.961901 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.961716    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.962424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.962349    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.964234 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.964211    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.964331 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.964231    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:53.970021 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:53.970000    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mzg7\" (UniqueName: \"kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7\") pod \"maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:54.071876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.071839    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:54.233135 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.233097    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:54.335367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.335273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:54.361055 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.361026    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:50:54.362799 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:50:54.362770    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f4b90e5_672e_4ed3_a9d8_19c636766d42.slice/crio-4740f50213eabe285afb3f087e5157c1e43b357e397785b1d601ae7d0e68e18c WatchSource:0}: Error finding container 4740f50213eabe285afb3f087e5157c1e43b357e397785b1d601ae7d0e68e18c: Status 404 returned error can't find the container with id 4740f50213eabe285afb3f087e5157c1e43b357e397785b1d601ae7d0e68e18c
Sep 08 13:50:54.435338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.435296    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:50:54.870830 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:54.870803    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" event={"ID":"5f4b90e5-672e-4ed3-a9d8-19c636766d42","Type":"ContainerStarted","Data":"4740f50213eabe285afb3f087e5157c1e43b357e397785b1d601ae7d0e68e18c"}
Sep 08 13:50:55.879050 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:55.878950    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" event={"ID":"5f4b90e5-672e-4ed3-a9d8-19c636766d42","Type":"ContainerStarted","Data":"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"}
Sep 08 13:50:55.879050 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:55.879027    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:50:55.897407 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:55.897356    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" podStartSLOduration=2.438432922 podStartE2EDuration="2.897339097s" totalImagesPullingTime="458.906175ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:50:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:50:55.89582465 +0000 UTC m=+1342.113740978" watchObservedRunningTime="2026-09-08 13:50:55.897339097 +0000 UTC m=+1342.115255455"
Sep 08 13:50:57.270914 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:50:57.270873    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:00.986431 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:00.986392    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:01.244584 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:01.244487    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:51:01.890607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:01.890545    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:51:14.726716 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:14.726654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:17.466521 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:17.466480    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:17.742732 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:17.742629    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:24.439021 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:24.438983    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:51:28.805806 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:28.805771    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:31.836908 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.836872    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:51:31.842255 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.842229    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.845789 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.845762    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2ee58a5f\"/\"kube-root-ca.crt\""
Sep 08 13:51:31.845936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.845816    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2ee58a5f\"/\"default-dockercfg-9gn79\""
Sep 08 13:51:31.845936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.845768    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-2ee58a5f\"/\"openshift-service-ca.crt\""
Sep 08 13:51:31.846051 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.845762    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-2ee58a5f\"/\"e2e-lifecycle-model-2ee58a5f-kserve-self-signed-certs\""
Sep 08 13:51:31.847920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.847893    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:51:31.868402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.868370    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:31.958920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.958870    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.959118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.958936    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.959118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.958958    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.959118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.959002    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.959118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.959074    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6d2pm\" (UniqueName: \"kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:31.959290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:31.959136    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.059841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.059784    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.059841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.059845    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.059898    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.059952    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6d2pm\" (UniqueName: \"kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.060023    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.060059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060360 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.060336    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060415 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.060378    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.060460 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.060447    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.062375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.062356    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.062536 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.062521    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.068954 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.068929    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6d2pm\" (UniqueName: \"kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm\") pod \"e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") " pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.156440 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.156342    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:32.435902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:32.435867    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:51:32.438463 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:51:32.438432    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0e4b191_cde5_418b_8321_3bd8805921e8.slice/crio-8a729a144bfd5981baaba63945aa84eb7d40fd00df434a77287107c5b38c73e0 WatchSource:0}: Error finding container 8a729a144bfd5981baaba63945aa84eb7d40fd00df434a77287107c5b38c73e0: Status 404 returned error can't find the container with id 8a729a144bfd5981baaba63945aa84eb7d40fd00df434a77287107c5b38c73e0
Sep 08 13:51:33.122522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:33.122486    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" event={"ID":"f0e4b191-cde5-418b-8321-3bd8805921e8","Type":"ContainerStarted","Data":"a19a31623d5a8daee7800bd2511abe870cbbb4cb24cbcea8c559023940573ce5"}
Sep 08 13:51:33.122522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:33.122527    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" event={"ID":"f0e4b191-cde5-418b-8321-3bd8805921e8","Type":"ContainerStarted","Data":"8a729a144bfd5981baaba63945aa84eb7d40fd00df434a77287107c5b38c73e0"}
Sep 08 13:51:33.141862 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:33.141807    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" podStartSLOduration=2.141789403 podStartE2EDuration="2.141789403s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:51:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:51:33.138711508 +0000 UTC m=+1379.356627842" watchObservedRunningTime="2026-09-08 13:51:33.141789403 +0000 UTC m=+1379.359705729"
Sep 08 13:51:38.339160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:38.339121    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:51:42.157296 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:42.157255    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:42.157745 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:42.157320    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:42.170048 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:42.170020    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:42.193062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:42.193032    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:51:53.338651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:53.338613    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:51:58.044290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:58.044247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:51:58.128062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:51:58.128026    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:15.538943 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:15.538905    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:52:22.110377 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:22.110336    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:28.714253 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.714164    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:52:28.714755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.714402    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" podUID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" containerName="maas-api" containerID="cri-o://df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a" gracePeriod=30
Sep 08 13:52:28.741721 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.741684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:28.897080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.896997    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:28.975413 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.975351    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:52:28.997205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:28.997092    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:29.035047 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.035007    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls\") pod \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") "
Sep 08 13:52:29.035248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.035073    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs\") pod \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") "
Sep 08 13:52:29.035248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.035128    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mzg7\" (UniqueName: \"kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7\") pod \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") "
Sep 08 13:52:29.035248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.035176    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca\") pod \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\" (UID: \"5f4b90e5-672e-4ed3-a9d8-19c636766d42\") "
Sep 08 13:52:29.035630 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.035582    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5f4b90e5-672e-4ed3-a9d8-19c636766d42" (UID: "5f4b90e5-672e-4ed3-a9d8-19c636766d42"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:52:29.037425 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.037392    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5f4b90e5-672e-4ed3-a9d8-19c636766d42" (UID: "5f4b90e5-672e-4ed3-a9d8-19c636766d42"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:52:29.037517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.037417    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5f4b90e5-672e-4ed3-a9d8-19c636766d42" (UID: "5f4b90e5-672e-4ed3-a9d8-19c636766d42"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:52:29.037517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.037475    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7" (OuterVolumeSpecName: "kube-api-access-5mzg7") pod "5f4b90e5-672e-4ed3-a9d8-19c636766d42" (UID: "5f4b90e5-672e-4ed3-a9d8-19c636766d42"). InnerVolumeSpecName "kube-api-access-5mzg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:52:29.067383 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.067347    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:29.136495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.136458    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mzg7\" (UniqueName: \"kubernetes.io/projected/5f4b90e5-672e-4ed3-a9d8-19c636766d42-kube-api-access-5mzg7\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.136495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.136489    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5f4b90e5-672e-4ed3-a9d8-19c636766d42-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.136495 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.136499    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.136786 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.136511    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5f4b90e5-672e-4ed3-a9d8-19c636766d42-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:29.478732 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.478692    2583 generic.go:356] "Generic (PLEG): container finished" podID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" containerID="df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a" exitCode=0
Sep 08 13:52:29.478942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.478740    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"
Sep 08 13:52:29.478942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.478781    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" event={"ID":"5f4b90e5-672e-4ed3-a9d8-19c636766d42","Type":"ContainerDied","Data":"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"}
Sep 08 13:52:29.478942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.478836    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj" event={"ID":"5f4b90e5-672e-4ed3-a9d8-19c636766d42","Type":"ContainerDied","Data":"4740f50213eabe285afb3f087e5157c1e43b357e397785b1d601ae7d0e68e18c"}
Sep 08 13:52:29.478942 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.478868    2583 scope.go:117] "RemoveContainer" containerID="df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"
Sep 08 13:52:29.490939 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.490913    2583 scope.go:117] "RemoveContainer" containerID="df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"
Sep 08 13:52:29.491256 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:52:29.491234    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a\": container with ID starting with df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a not found: ID does not exist" containerID="df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"
Sep 08 13:52:29.491334 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.491269    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a"} err="failed to get container status \"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a\": rpc error: code = NotFound desc = could not find container \"df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a\": container with ID starting with df6367c531b15b235816b66c3b1499c3ebf93884a6293d86e089c894dedd804a not found: ID does not exist"
Sep 08 13:52:29.503739 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.503705    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:52:29.507612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:29.507590    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-2ee58a5f-7fb5f7b6cf-6dnkj"]
Sep 08 13:52:30.232307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:30.232278    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" path="/var/lib/kubelet/pods/5f4b90e5-672e-4ed3-a9d8-19c636766d42/volumes"
Sep 08 13:52:32.435803 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:32.435772    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:33.143121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:33.143077    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:52:37.420835 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:37.420799    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:52:37.517156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:37.517120    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:52:48.238864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:48.238830    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:52:52.442078 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.442014    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" podUID="f0e4b191-cde5-418b-8321-3bd8805921e8" containerName="main" containerID="cri-o://a19a31623d5a8daee7800bd2511abe870cbbb4cb24cbcea8c559023940573ce5" gracePeriod=45
Sep 08 13:52:52.631717 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.631662    2583 generic.go:356] "Generic (PLEG): container finished" podID="f0e4b191-cde5-418b-8321-3bd8805921e8" containerID="a19a31623d5a8daee7800bd2511abe870cbbb4cb24cbcea8c559023940573ce5" exitCode=0
Sep 08 13:52:52.631905 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.631707    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" event={"ID":"f0e4b191-cde5-418b-8321-3bd8805921e8","Type":"ContainerDied","Data":"a19a31623d5a8daee7800bd2511abe870cbbb4cb24cbcea8c559023940573ce5"}
Sep 08 13:52:52.702182 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.702107    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:52:52.875635 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875578    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.875860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875669    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6d2pm\" (UniqueName: \"kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.875860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875708    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.875860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875725    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.875860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875760    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.875860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875822    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home\") pod \"f0e4b191-cde5-418b-8321-3bd8805921e8\" (UID: \"f0e4b191-cde5-418b-8321-3bd8805921e8\") "
Sep 08 13:52:52.876128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.875995    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:52:52.876128 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.876004    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache" (OuterVolumeSpecName: "model-cache") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:52:52.876235 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.876196    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home" (OuterVolumeSpecName: "home") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:52:52.876294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.876247    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:52.876294 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.876267    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:52.878260 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.878225    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm" (OuterVolumeSpecName: "dshm") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:52:52.878398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.878317    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:52:52.878398 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.878376    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm" (OuterVolumeSpecName: "kube-api-access-6d2pm") pod "f0e4b191-cde5-418b-8321-3bd8805921e8" (UID: "f0e4b191-cde5-418b-8321-3bd8805921e8"). InnerVolumeSpecName "kube-api-access-6d2pm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:52:52.977424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.977333    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:52.977424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.977367    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6d2pm\" (UniqueName: \"kubernetes.io/projected/f0e4b191-cde5-418b-8321-3bd8805921e8-kube-api-access-6d2pm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:52.977424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.977378    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0e4b191-cde5-418b-8321-3bd8805921e8-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:52.977424 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:52.977388    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f0e4b191-cde5-418b-8321-3bd8805921e8-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:52:53.639620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:53.639579    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc" event={"ID":"f0e4b191-cde5-418b-8321-3bd8805921e8","Type":"ContainerDied","Data":"8a729a144bfd5981baaba63945aa84eb7d40fd00df434a77287107c5b38c73e0"}
Sep 08 13:52:53.640146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:53.639636    2583 scope.go:117] "RemoveContainer" containerID="a19a31623d5a8daee7800bd2511abe870cbbb4cb24cbcea8c559023940573ce5"
Sep 08 13:52:53.640146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:53.639596    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"
Sep 08 13:52:53.662017 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:53.661982    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:52:53.666344 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:53.666315    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-2ee58a5f/e2e-lifecycle-model-2ee58a5f-kserve-59c64668c-5qdbc"]
Sep 08 13:52:54.242478 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:52:54.242439    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0e4b191-cde5-418b-8321-3bd8805921e8" path="/var/lib/kubelet/pods/f0e4b191-cde5-418b-8321-3bd8805921e8/volumes"
Sep 08 13:53:04.933274 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:04.933233    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:53:13.675097 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:13.675062    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:14.692802 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:14.692763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:19.339152 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:19.339116    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:53:20.589708 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.589668    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:20.693368 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.693294    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:20.844113 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.844027    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:20.881546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.881513    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7720-52434c"]
Sep 08 13:53:20.882004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.881990    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0e4b191-cde5-418b-8321-3bd8805921e8" containerName="main"
Sep 08 13:53:20.882071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.882006    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0e4b191-cde5-418b-8321-3bd8805921e8" containerName="main"
Sep 08 13:53:20.882071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.882025    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" containerName="maas-api"
Sep 08 13:53:20.882071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.882030    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" containerName="maas-api"
Sep 08 13:53:20.882170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.882110    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f4b90e5-672e-4ed3-a9d8-19c636766d42" containerName="maas-api"
Sep 08 13:53:20.882170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.882120    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0e4b191-cde5-418b-8321-3bd8805921e8" containerName="main"
Sep 08 13:53:20.884947 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.884926    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:20.887292 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.887264    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qzs56\""
Sep 08 13:53:20.890327 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.890301    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-52434c"]
Sep 08 13:53:20.963172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:20.963134    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rx6k\" (UniqueName: \"kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k\") pod \"mt-curl-7720-52434c\" (UID: \"10a72410-5cdf-4887-ba2c-ad89580f596b\") " pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:21.064007 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:21.063969    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rx6k\" (UniqueName: \"kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k\") pod \"mt-curl-7720-52434c\" (UID: \"10a72410-5cdf-4887-ba2c-ad89580f596b\") " pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:21.072957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:21.072930    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rx6k\" (UniqueName: \"kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k\") pod \"mt-curl-7720-52434c\" (UID: \"10a72410-5cdf-4887-ba2c-ad89580f596b\") " pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:21.197663 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:21.197546    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:21.471095 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:21.471064    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-52434c"]
Sep 08 13:53:21.473085 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:53:21.473056    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod10a72410_5cdf_4887_ba2c_ad89580f596b.slice/crio-7195ed70db5a18df0631e76538bce99c2b20c580cee7342ae250d64497a9d7f1 WatchSource:0}: Error finding container 7195ed70db5a18df0631e76538bce99c2b20c580cee7342ae250d64497a9d7f1: Status 404 returned error can't find the container with id 7195ed70db5a18df0631e76538bce99c2b20c580cee7342ae250d64497a9d7f1
Sep 08 13:53:21.831935 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:21.831896    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-52434c" event={"ID":"10a72410-5cdf-4887-ba2c-ad89580f596b","Type":"ContainerStarted","Data":"7195ed70db5a18df0631e76538bce99c2b20c580cee7342ae250d64497a9d7f1"}
Sep 08 13:53:22.109871 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.109788    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:22.649713 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.649681    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:22.749759 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.749722    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:22.840189 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.840159    2583 generic.go:356] "Generic (PLEG): container finished" podID="10a72410-5cdf-4887-ba2c-ad89580f596b" containerID="7a7aee259428d5527b99e6702d2a5c3c6b3749a508996149f57e2f49be47500e" exitCode=0
Sep 08 13:53:22.840623 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.840235    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-52434c" event={"ID":"10a72410-5cdf-4887-ba2c-ad89580f596b","Type":"ContainerDied","Data":"7a7aee259428d5527b99e6702d2a5c3c6b3749a508996149f57e2f49be47500e"}
Sep 08 13:53:22.899066 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.899014    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:22.991691 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:22.991609    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:23.094644 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:23.094601    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:23.987232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:23.987203    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:24.006876 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.006844    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7720-52434c_10a72410-5cdf-4887-ba2c-ad89580f596b/mt-curl-7720-52434c/0.log"
Sep 08 13:53:24.030851 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.030815    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7720-52434c"]
Sep 08 13:53:24.034540 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.034510    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7720-52434c"]
Sep 08 13:53:24.097932 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.097837    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rx6k\" (UniqueName: \"kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k\") pod \"10a72410-5cdf-4887-ba2c-ad89580f596b\" (UID: \"10a72410-5cdf-4887-ba2c-ad89580f596b\") "
Sep 08 13:53:24.100086 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.100057    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k" (OuterVolumeSpecName: "kube-api-access-6rx6k") pod "10a72410-5cdf-4887-ba2c-ad89580f596b" (UID: "10a72410-5cdf-4887-ba2c-ad89580f596b"). InnerVolumeSpecName "kube-api-access-6rx6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:53:24.199178 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.199135    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rx6k\" (UniqueName: \"kubernetes.io/projected/10a72410-5cdf-4887-ba2c-ad89580f596b-kube-api-access-6rx6k\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:53:24.232840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.232803    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10a72410-5cdf-4887-ba2c-ad89580f596b" path="/var/lib/kubelet/pods/10a72410-5cdf-4887-ba2c-ad89580f596b/volumes"
Sep 08 13:53:24.294903 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.294868    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:24.855752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.855728    2583 scope.go:117] "RemoveContainer" containerID="7a7aee259428d5527b99e6702d2a5c3c6b3749a508996149f57e2f49be47500e"
Sep 08 13:53:24.856064 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:24.855730    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-52434c"
Sep 08 13:53:25.796198 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:25.796168    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:26.452322 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:26.452286    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:30.134435 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.134392    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:53:30.135156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.135137    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10a72410-5cdf-4887-ba2c-ad89580f596b" containerName="mt-curl-7720-52434c"
Sep 08 13:53:30.135156 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.135157    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="10a72410-5cdf-4887-ba2c-ad89580f596b" containerName="mt-curl-7720-52434c"
Sep 08 13:53:30.135327 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.135232    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="10a72410-5cdf-4887-ba2c-ad89580f596b" containerName="mt-curl-7720-52434c"
Sep 08 13:53:30.139976 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.139954    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.143578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.143139    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1f082e\"/\"default-dockercfg-22wdv\""
Sep 08 13:53:30.143578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.143258    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1f082e\"/\"kube-root-ca.crt\""
Sep 08 13:53:30.143578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.143264    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-1f082e\"/\"sub-test-model-1f082e-kserve-self-signed-certs\""
Sep 08 13:53:30.143578 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.143295    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-1f082e\"/\"openshift-service-ca.crt\""
Sep 08 13:53:30.148018 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.147995    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:53:30.265625 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265586    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.265808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265635    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.265808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265677    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gqjq5\" (UniqueName: \"kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.265808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265749    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.265808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265779    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.265808 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.265797    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366373 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366331    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366380    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366518    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gqjq5\" (UniqueName: \"kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366607    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366628    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366659 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366648    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366823 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366768    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366879 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.366975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.366951    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.369150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.369129    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.369300 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.369283    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.382418 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.382390    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gqjq5\" (UniqueName: \"kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5\") pod \"sub-test-model-1f082e-kserve-6d74cdb955-wtcql\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") " pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.449620 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.449527    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:30.452510 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.452483    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:30.537822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.537791    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:30.728248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.728218    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:53:30.729321 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:53:30.729299    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfadddc1c_e45c_4f94_bcbe_3120cf495b49.slice/crio-080db37058051c63f2437526ff47ee7085c88c3d759fe0bfcc9506ed217ed123 WatchSource:0}: Error finding container 080db37058051c63f2437526ff47ee7085c88c3d759fe0bfcc9506ed217ed123: Status 404 returned error can't find the container with id 080db37058051c63f2437526ff47ee7085c88c3d759fe0bfcc9506ed217ed123
Sep 08 13:53:30.897545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.897510    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" event={"ID":"fadddc1c-e45c-4f94-bcbe-3120cf495b49","Type":"ContainerStarted","Data":"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"}
Sep 08 13:53:30.897545 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.897575    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" event={"ID":"fadddc1c-e45c-4f94-bcbe-3120cf495b49","Type":"ContainerStarted","Data":"080db37058051c63f2437526ff47ee7085c88c3d759fe0bfcc9506ed217ed123"}
Sep 08 13:53:30.915418 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:30.915356    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" podStartSLOduration=0.915341153 podStartE2EDuration="915.341153ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:53:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:53:30.912909662 +0000 UTC m=+1497.130825989" watchObservedRunningTime="2026-09-08 13:53:30.915341153 +0000 UTC m=+1497.133257478"
Sep 08 13:53:31.064777 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:31.064745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:34.335457 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:34.335430    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:53:34.339355 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:34.339335    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:53:36.547166 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:36.547124    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:53:37.190027 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:37.189978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:37.285934 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:37.285896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:37.388465 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:37.388420    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:40.453376 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:40.453340    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:40.453827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:40.453481    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:40.466494 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:40.466466    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:40.975423 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:40.975386    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:53:47.919433 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:47.919389    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:48.173139 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:48.173032    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:51.262259 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:51.262226    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:53:58.035849 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:53:58.035763    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:54:09.227479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:09.227436    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:14.948731 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:14.948684    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:15.048983 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:15.048948    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:15.363828 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:15.363798    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:15.780812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:15.780780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:15.999275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:15.999237    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:18.627326 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:18.627293    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:18.724112 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:18.724076    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:19.848273 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.848238    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:54:19.852125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.852104    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.854724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.854697    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-569975-serving-cert\""
Sep 08 13:54:19.858605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.858577    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.858752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.858648    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m678g\" (UniqueName: \"kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.858752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.858693    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.858875 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.858808    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.866186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.866158    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:54:19.879014 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.878982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:19.960085 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.960049    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.960306 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.960094    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m678g\" (UniqueName: \"kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.960306 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.960130    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.960306 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.960198    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.960479 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:19.960340    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-569975-serving-cert: secret "maas-api-e2e-api-a-569975-serving-cert" not found
Sep 08 13:54:19.960479 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:19.960410    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls podName:80a6ed4a-9435-4f2c-9962-0017d270706e nodeName:}" failed. No retries permitted until 2026-09-08 13:54:20.460389937 +0000 UTC m=+1546.678306259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls") pod "maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" (UID: "80a6ed4a-9435-4f2c-9962-0017d270706e") : secret "maas-api-e2e-api-a-569975-serving-cert" not found
Sep 08 13:54:19.960955 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.960934    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.962925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.962900    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.969387 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.969364    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m678g\" (UniqueName: \"kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:19.979130 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:19.979104    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:20.168206 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.168093    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:20.352896 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.352865    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:20.451908 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.451819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:20.465668 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.465633    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:20.468501 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.468473    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") pod \"maas-api-e2e-api-a-569975-7557f76dbb-z2tr7\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:20.765098 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:20.765007    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:21.041414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:21.041376    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:54:21.043353 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:54:21.043322    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80a6ed4a_9435_4f2c_9962_0017d270706e.slice/crio-2d2439921bc20b49affa4a457ba2785bde7b44aab5a4865725b3a619c0332cf5 WatchSource:0}: Error finding container 2d2439921bc20b49affa4a457ba2785bde7b44aab5a4865725b3a619c0332cf5: Status 404 returned error can't find the container with id 2d2439921bc20b49affa4a457ba2785bde7b44aab5a4865725b3a619c0332cf5
Sep 08 13:54:21.226039 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:21.226000    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" event={"ID":"80a6ed4a-9435-4f2c-9962-0017d270706e","Type":"ContainerStarted","Data":"2d2439921bc20b49affa4a457ba2785bde7b44aab5a4865725b3a619c0332cf5"}
Sep 08 13:54:21.632777 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:21.632743    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:54:22.233372 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:22.233333    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" event={"ID":"80a6ed4a-9435-4f2c-9962-0017d270706e","Type":"ContainerStarted","Data":"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"}
Sep 08 13:54:22.233844 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:22.233400    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:22.257993 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:22.257936    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" podStartSLOduration=2.873673197 podStartE2EDuration="3.257918005s" totalImagesPullingTime="384.244808ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:22.254227208 +0000 UTC m=+1548.472143550" watchObservedRunningTime="2026-09-08 13:54:22.257918005 +0000 UTC m=+1548.475834333"
Sep 08 13:54:25.893537 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:25.893497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:25.993748 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:25.993699    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:28.244530 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:28.244495    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:54:36.215323 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.215283    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7720-0cb359"]
Sep 08 13:54:36.218985 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.218968    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:36.221237 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.221209    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-qzs56\""
Sep 08 13:54:36.234959 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.234935    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-0cb359"]
Sep 08 13:54:36.344542 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.344501    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwv2t\" (UniqueName: \"kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t\") pod \"mt-curl-7720-0cb359\" (UID: \"71b2b1d6-9032-490d-81a7-2495ff01903f\") " pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:36.445610 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.445575    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwv2t\" (UniqueName: \"kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t\") pod \"mt-curl-7720-0cb359\" (UID: \"71b2b1d6-9032-490d-81a7-2495ff01903f\") " pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:36.454079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.454053    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwv2t\" (UniqueName: \"kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t\") pod \"mt-curl-7720-0cb359\" (UID: \"71b2b1d6-9032-490d-81a7-2495ff01903f\") " pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:36.530622 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.530589    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:36.805125 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:36.805098    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-0cb359"]
Sep 08 13:54:36.807418 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:54:36.807380    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod71b2b1d6_9032_490d_81a7_2495ff01903f.slice/crio-3b7064ab2efd9cb34b073fa4871cc43741bc9c1be49c460f716d460acffd88e4 WatchSource:0}: Error finding container 3b7064ab2efd9cb34b073fa4871cc43741bc9c1be49c460f716d460acffd88e4: Status 404 returned error can't find the container with id 3b7064ab2efd9cb34b073fa4871cc43741bc9c1be49c460f716d460acffd88e4
Sep 08 13:54:37.331782 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:37.331739    2583 generic.go:356] "Generic (PLEG): container finished" podID="71b2b1d6-9032-490d-81a7-2495ff01903f" containerID="09957b1a05d88554ac80a1321432497de27165326a3c24005023d3abec3525bb" exitCode=0
Sep 08 13:54:37.332186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:37.331835    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-0cb359" event={"ID":"71b2b1d6-9032-490d-81a7-2495ff01903f","Type":"ContainerDied","Data":"09957b1a05d88554ac80a1321432497de27165326a3c24005023d3abec3525bb"}
Sep 08 13:54:37.332186 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:37.331872    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-0cb359" event={"ID":"71b2b1d6-9032-490d-81a7-2495ff01903f","Type":"ContainerStarted","Data":"3b7064ab2efd9cb34b073fa4871cc43741bc9c1be49c460f716d460acffd88e4"}
Sep 08 13:54:38.487722 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.487693    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:38.507301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.507272    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7720-0cb359_71b2b1d6-9032-490d-81a7-2495ff01903f/mt-curl-7720-0cb359/0.log"
Sep 08 13:54:38.531920 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.531884    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7720-0cb359"]
Sep 08 13:54:38.537902 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.537873    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7720-0cb359"]
Sep 08 13:54:38.568375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.568339    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwv2t\" (UniqueName: \"kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t\") pod \"71b2b1d6-9032-490d-81a7-2495ff01903f\" (UID: \"71b2b1d6-9032-490d-81a7-2495ff01903f\") "
Sep 08 13:54:38.570691 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.570665    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t" (OuterVolumeSpecName: "kube-api-access-jwv2t") pod "71b2b1d6-9032-490d-81a7-2495ff01903f" (UID: "71b2b1d6-9032-490d-81a7-2495ff01903f"). InnerVolumeSpecName "kube-api-access-jwv2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:54:38.660978 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.660893    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/mt-curl-7720-b7d48d"]
Sep 08 13:54:38.661376 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.661363    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71b2b1d6-9032-490d-81a7-2495ff01903f" containerName="mt-curl-7720-0cb359"
Sep 08 13:54:38.661420 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.661378    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="71b2b1d6-9032-490d-81a7-2495ff01903f" containerName="mt-curl-7720-0cb359"
Sep 08 13:54:38.661472 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.661462    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="71b2b1d6-9032-490d-81a7-2495ff01903f" containerName="mt-curl-7720-0cb359"
Sep 08 13:54:38.664458 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.664442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:38.669311 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.669282    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-b7d48d"]
Sep 08 13:54:38.669442 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.669362    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwv2t\" (UniqueName: \"kubernetes.io/projected/71b2b1d6-9032-490d-81a7-2495ff01903f-kube-api-access-jwv2t\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:38.770447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.770409    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx4zd\" (UniqueName: \"kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd\") pod \"mt-curl-7720-b7d48d\" (UID: \"6daa5793-0bb1-44b1-8bec-003d3af6843d\") " pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:38.871864 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.871826    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lx4zd\" (UniqueName: \"kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd\") pod \"mt-curl-7720-b7d48d\" (UID: \"6daa5793-0bb1-44b1-8bec-003d3af6843d\") " pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:38.880726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.880698    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx4zd\" (UniqueName: \"kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd\") pod \"mt-curl-7720-b7d48d\" (UID: \"6daa5793-0bb1-44b1-8bec-003d3af6843d\") " pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:38.976090 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:38.975992    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:39.252600 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:39.252412    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/mt-curl-7720-b7d48d"]
Sep 08 13:54:39.255212 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:54:39.255185    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6daa5793_0bb1_44b1_8bec_003d3af6843d.slice/crio-295f997e4fe5ddf2c2126249489276f2e6fea481fca79d2c0a11471b1ef29263 WatchSource:0}: Error finding container 295f997e4fe5ddf2c2126249489276f2e6fea481fca79d2c0a11471b1ef29263: Status 404 returned error can't find the container with id 295f997e4fe5ddf2c2126249489276f2e6fea481fca79d2c0a11471b1ef29263
Sep 08 13:54:39.347190 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:39.347146    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-b7d48d" event={"ID":"6daa5793-0bb1-44b1-8bec-003d3af6843d","Type":"ContainerStarted","Data":"295f997e4fe5ddf2c2126249489276f2e6fea481fca79d2c0a11471b1ef29263"}
Sep 08 13:54:39.348686 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:39.348666    2583 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b7064ab2efd9cb34b073fa4871cc43741bc9c1be49c460f716d460acffd88e4"
Sep 08 13:54:39.348789 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:39.348720    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:39.359714 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:39.359688    2583 status_manager.go:1018] "Failed to get status for pod" err="pods \"mt-curl-7720-0cb359\" is forbidden: User \"system:node:ip-10-0-140-23.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-140-23.ec2.internal' and this object" podUID="71b2b1d6-9032-490d-81a7-2495ff01903f" pod="opendatahub/mt-curl-7720-0cb359"
Sep 08 13:54:40.232367 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:40.232330    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71b2b1d6-9032-490d-81a7-2495ff01903f" path="/var/lib/kubelet/pods/71b2b1d6-9032-490d-81a7-2495ff01903f/volumes"
Sep 08 13:54:40.356072 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:40.356035    2583 generic.go:356] "Generic (PLEG): container finished" podID="6daa5793-0bb1-44b1-8bec-003d3af6843d" containerID="a24cf156324d0145a0db814eff195f88ec3e1d1c520da4b18a3e0a3c62da2089" exitCode=0
Sep 08 13:54:40.356266 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:40.356119    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/mt-curl-7720-b7d48d" event={"ID":"6daa5793-0bb1-44b1-8bec-003d3af6843d","Type":"ContainerDied","Data":"a24cf156324d0145a0db814eff195f88ec3e1d1c520da4b18a3e0a3c62da2089"}
Sep 08 13:54:41.508301 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.508274    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:41.527089 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.527043    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_mt-curl-7720-b7d48d_6daa5793-0bb1-44b1-8bec-003d3af6843d/mt-curl-7720-b7d48d/0.log"
Sep 08 13:54:41.550746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.550704    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/mt-curl-7720-b7d48d"]
Sep 08 13:54:41.554597 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.554571    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/mt-curl-7720-b7d48d"]
Sep 08 13:54:41.600566 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.600521    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx4zd\" (UniqueName: \"kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd\") pod \"6daa5793-0bb1-44b1-8bec-003d3af6843d\" (UID: \"6daa5793-0bb1-44b1-8bec-003d3af6843d\") "
Sep 08 13:54:41.602763 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.602732    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd" (OuterVolumeSpecName: "kube-api-access-lx4zd") pod "6daa5793-0bb1-44b1-8bec-003d3af6843d" (UID: "6daa5793-0bb1-44b1-8bec-003d3af6843d"). InnerVolumeSpecName "kube-api-access-lx4zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:54:41.701444 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:41.701360    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx4zd\" (UniqueName: \"kubernetes.io/projected/6daa5793-0bb1-44b1-8bec-003d3af6843d-kube-api-access-lx4zd\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:42.232131 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:42.232095    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6daa5793-0bb1-44b1-8bec-003d3af6843d" path="/var/lib/kubelet/pods/6daa5793-0bb1-44b1-8bec-003d3af6843d/volumes"
Sep 08 13:54:42.369891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:42.369860    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/mt-curl-7720-b7d48d"
Sep 08 13:54:42.369891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:42.369877    2583 scope.go:117] "RemoveContainer" containerID="a24cf156324d0145a0db814eff195f88ec3e1d1c520da4b18a3e0a3c62da2089"
Sep 08 13:54:42.938262 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:42.938229    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:54:43.577075 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:43.577017    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:54:43.597340 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:43.597310    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:43.664153 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:43.664118    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:44.100216 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:44.100186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:44.414056 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:44.413959    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:44.565499 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:44.565462    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:45.014847 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:45.014814    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:47.206651 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:47.206621    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:47.304538 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:47.304503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:48.353774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.353743    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:54:48.354232 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.354218    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6daa5793-0bb1-44b1-8bec-003d3af6843d" containerName="mt-curl-7720-b7d48d"
Sep 08 13:54:48.354290 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.354234    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="6daa5793-0bb1-44b1-8bec-003d3af6843d" containerName="mt-curl-7720-b7d48d"
Sep 08 13:54:48.354331 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.354305    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="6daa5793-0bb1-44b1-8bec-003d3af6843d" containerName="mt-curl-7720-b7d48d"
Sep 08 13:54:48.358972 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.358948    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.361205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.361186    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-8ef041-serving-cert\""
Sep 08 13:54:48.362666 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.362641    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.362760 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.362682    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.362760 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.362709    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2x5mk\" (UniqueName: \"kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.362868 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.362837    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.371353 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.371328    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:54:48.384358 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.384325    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:48.446483 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.446438    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:48.463657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.463622    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.463657 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.463659    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.463852 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.463682    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2x5mk\" (UniqueName: \"kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.463852 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.463770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.463952 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:48.463934    2583 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-serving-cert: secret "maas-api-e2e-api-b-8ef041-serving-cert" not found
Sep 08 13:54:48.464034 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:48.464021    2583 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls podName:772b736c-98bf-4983-91eb-7e76f0fffec8 nodeName:}" failed. No retries permitted until 2026-09-08 13:54:48.963999188 +0000 UTC m=+1575.181915490 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls") pod "maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" (UID: "772b736c-98bf-4983-91eb-7e76f0fffec8") : secret "maas-api-e2e-api-b-8ef041-serving-cert" not found
Sep 08 13:54:48.464412 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.464389    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.466432 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.466408    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.481655 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.481627    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2x5mk\" (UniqueName: \"kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.692720 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.692624    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:48.910778 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.910745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:48.967298 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.967207    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.970248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.970218    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") pod \"maas-api-e2e-api-b-8ef041-7df8d77c8-zk849\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:48.972174 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:48.972156    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:49.011809 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:49.011745    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:49.266202 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:49.266168    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:54:49.267465 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:54:49.267439    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod772b736c_98bf_4983_91eb_7e76f0fffec8.slice/crio-fff9b944a9005ded5ff3660831bed5eab131e3c0e90cb9b338d3f63a9da323a1 WatchSource:0}: Error finding container fff9b944a9005ded5ff3660831bed5eab131e3c0e90cb9b338d3f63a9da323a1: Status 404 returned error can't find the container with id fff9b944a9005ded5ff3660831bed5eab131e3c0e90cb9b338d3f63a9da323a1
Sep 08 13:54:49.420211 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:49.420174    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" event={"ID":"772b736c-98bf-4983-91eb-7e76f0fffec8","Type":"ContainerStarted","Data":"fff9b944a9005ded5ff3660831bed5eab131e3c0e90cb9b338d3f63a9da323a1"}
Sep 08 13:54:50.428752 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:50.428710    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" event={"ID":"772b736c-98bf-4983-91eb-7e76f0fffec8","Type":"ContainerStarted","Data":"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"}
Sep 08 13:54:50.429200 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:50.428825    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:50.448137 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:50.448088    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" podStartSLOduration=2.017457717 podStartE2EDuration="2.448073314s" totalImagesPullingTime="430.615597ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:54:50.445675164 +0000 UTC m=+1576.663591508" watchObservedRunningTime="2026-09-08 13:54:50.448073314 +0000 UTC m=+1576.665989671"
Sep 08 13:54:51.897769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:51.897734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:55.662310 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:55.662269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:54:56.440848 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:56.440816    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:54:58.600765 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.600727    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" podUID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" containerName="main" containerID="cri-o://dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff" gracePeriod=45
Sep 08 13:54:58.863768 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.863700    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:54:58.966960 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.966921    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967136 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.966970    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967136 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967063    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967136 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967118    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gqjq5\" (UniqueName: \"kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967185    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967271 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967215    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir\") pod \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\" (UID: \"fadddc1c-e45c-4f94-bcbe-3120cf495b49\") "
Sep 08 13:54:58.967363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967293    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache" (OuterVolumeSpecName: "model-cache") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:54:58.967533 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967490    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home" (OuterVolumeSpecName: "home") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:54:58.967667 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967508    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:54:58.967667 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967651    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:58.967936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967671    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:58.967936 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.967687    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:58.969468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.969446    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:54:58.969468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.969450    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm" (OuterVolumeSpecName: "dshm") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:54:58.969635 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:58.969507    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5" (OuterVolumeSpecName: "kube-api-access-gqjq5") pod "fadddc1c-e45c-4f94-bcbe-3120cf495b49" (UID: "fadddc1c-e45c-4f94-bcbe-3120cf495b49"). InnerVolumeSpecName "kube-api-access-gqjq5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:54:59.069070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.069033    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gqjq5\" (UniqueName: \"kubernetes.io/projected/fadddc1c-e45c-4f94-bcbe-3120cf495b49-kube-api-access-gqjq5\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:59.069070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.069068    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fadddc1c-e45c-4f94-bcbe-3120cf495b49-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:59.069070 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.069087    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fadddc1c-e45c-4f94-bcbe-3120cf495b49-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:54:59.491735 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.491687    2583 generic.go:356] "Generic (PLEG): container finished" podID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" containerID="dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff" exitCode=0
Sep 08 13:54:59.491735 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.491727    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"
Sep 08 13:54:59.492002 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.491773    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" event={"ID":"fadddc1c-e45c-4f94-bcbe-3120cf495b49","Type":"ContainerDied","Data":"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"}
Sep 08 13:54:59.492002 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.491818    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql" event={"ID":"fadddc1c-e45c-4f94-bcbe-3120cf495b49","Type":"ContainerDied","Data":"080db37058051c63f2437526ff47ee7085c88c3d759fe0bfcc9506ed217ed123"}
Sep 08 13:54:59.492002 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.491849    2583 scope.go:117] "RemoveContainer" containerID="dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"
Sep 08 13:54:59.502358 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.502335    2583 scope.go:117] "RemoveContainer" containerID="dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"
Sep 08 13:54:59.502716 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:54:59.502679    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff\": container with ID starting with dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff not found: ID does not exist" containerID="dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"
Sep 08 13:54:59.502834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.502721    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff"} err="failed to get container status \"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff\": rpc error: code = NotFound desc = could not find container \"dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff\": container with ID starting with dbd4c9ec917876775e3530ac10f1740347743602b90a005d59c6d15238be73ff not found: ID does not exist"
Sep 08 13:54:59.513957 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.513928    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:54:59.519757 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:54:59.519728    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-1f082e/sub-test-model-1f082e-kserve-6d74cdb955-wtcql"]
Sep 08 13:55:00.043300 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:00.043258    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:55:00.233004 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:00.232974    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" path="/var/lib/kubelet/pods/fadddc1c-e45c-4f94-bcbe-3120cf495b49/volumes"
Sep 08 13:55:05.246629 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:05.246591    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:09.339540 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:09.339503    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:12.230605 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:12.230532    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:15.457416 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:15.457381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:22.072764 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:22.072721    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:25.577164 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:25.577055    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.135804 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.135768    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:55:31.136305 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.135974    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" podUID="80a6ed4a-9435-4f2c-9962-0017d270706e" containerName="maas-api" containerID="cri-o://2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a" gracePeriod=30
Sep 08 13:55:31.165025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.164978    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.231282 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.231247    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.331623 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.331583    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.391962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.391877    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:55:31.396866 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.396838    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.511382 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.511349    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m678g\" (UniqueName: \"kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g\") pod \"80a6ed4a-9435-4f2c-9962-0017d270706e\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") "
Sep 08 13:55:31.511569 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.511410    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs\") pod \"80a6ed4a-9435-4f2c-9962-0017d270706e\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") "
Sep 08 13:55:31.511569 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.511512    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") pod \"80a6ed4a-9435-4f2c-9962-0017d270706e\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") "
Sep 08 13:55:31.511569 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.511560    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca\") pod \"80a6ed4a-9435-4f2c-9962-0017d270706e\" (UID: \"80a6ed4a-9435-4f2c-9962-0017d270706e\") "
Sep 08 13:55:31.512024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.511996    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "80a6ed4a-9435-4f2c-9962-0017d270706e" (UID: "80a6ed4a-9435-4f2c-9962-0017d270706e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:31.513854 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.513828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "80a6ed4a-9435-4f2c-9962-0017d270706e" (UID: "80a6ed4a-9435-4f2c-9962-0017d270706e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:55:31.514170 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.514148    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "80a6ed4a-9435-4f2c-9962-0017d270706e" (UID: "80a6ed4a-9435-4f2c-9962-0017d270706e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:55:31.514241 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.514171    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g" (OuterVolumeSpecName: "kube-api-access-m678g") pod "80a6ed4a-9435-4f2c-9962-0017d270706e" (UID: "80a6ed4a-9435-4f2c-9962-0017d270706e"). InnerVolumeSpecName "kube-api-access-m678g". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:31.612813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.612779    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m678g\" (UniqueName: \"kubernetes.io/projected/80a6ed4a-9435-4f2c-9962-0017d270706e-kube-api-access-m678g\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.612813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.612809    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.612813 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.612818    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/80a6ed4a-9435-4f2c-9962-0017d270706e-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.613061 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.612828    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/80a6ed4a-9435-4f2c-9962-0017d270706e-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:31.711281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.711183    2583 generic.go:356] "Generic (PLEG): container finished" podID="80a6ed4a-9435-4f2c-9962-0017d270706e" containerID="2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a" exitCode=0
Sep 08 13:55:31.711281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.711237    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"
Sep 08 13:55:31.711281 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.711246    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" event={"ID":"80a6ed4a-9435-4f2c-9962-0017d270706e","Type":"ContainerDied","Data":"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"}
Sep 08 13:55:31.711592 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.711288    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7" event={"ID":"80a6ed4a-9435-4f2c-9962-0017d270706e","Type":"ContainerDied","Data":"2d2439921bc20b49affa4a457ba2785bde7b44aab5a4865725b3a619c0332cf5"}
Sep 08 13:55:31.711592 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.711310    2583 scope.go:117] "RemoveContainer" containerID="2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"
Sep 08 13:55:31.720924 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.720891    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:31.731819 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.731794    2583 scope.go:117] "RemoveContainer" containerID="2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"
Sep 08 13:55:31.732267 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:55:31.732241    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a\": container with ID starting with 2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a not found: ID does not exist" containerID="2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"
Sep 08 13:55:31.732366 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.732274    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a"} err="failed to get container status \"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a\": rpc error: code = NotFound desc = could not find container \"2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a\": container with ID starting with 2690e84d866d4f63868968915f60cfee4c1499b033d59933b4e4f97d7fb7919a not found: ID does not exist"
Sep 08 13:55:31.743529 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.743507    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:55:31.748522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:31.748501    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-569975-7557f76dbb-z2tr7"]
Sep 08 13:55:32.232270 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:32.232238    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80a6ed4a-9435-4f2c-9962-0017d270706e" path="/var/lib/kubelet/pods/80a6ed4a-9435-4f2c-9962-0017d270706e/volumes"
Sep 08 13:55:34.145445 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:34.145405    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:36.139257 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139223    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:55:36.139700 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139688    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80a6ed4a-9435-4f2c-9962-0017d270706e" containerName="maas-api"
Sep 08 13:55:36.139746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139702    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="80a6ed4a-9435-4f2c-9962-0017d270706e" containerName="maas-api"
Sep 08 13:55:36.139746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139719    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" containerName="main"
Sep 08 13:55:36.139746 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139725    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" containerName="main"
Sep 08 13:55:36.139841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139808    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="80a6ed4a-9435-4f2c-9962-0017d270706e" containerName="maas-api"
Sep 08 13:55:36.139841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.139819    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fadddc1c-e45c-4f94-bcbe-3120cf495b49" containerName="main"
Sep 08 13:55:36.143991 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.143969    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.146289 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.146266    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-905fec\"/\"default-dockercfg-fxtd9\""
Sep 08 13:55:36.146438 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.146305    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-905fec\"/\"kube-root-ca.crt\""
Sep 08 13:55:36.147117 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.147098    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-905fec\"/\"rate-test-model-905fec-kserve-self-signed-certs\""
Sep 08 13:55:36.147220 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.147194    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-905fec\"/\"openshift-service-ca.crt\""
Sep 08 13:55:36.155108 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.155075    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:55:36.243834 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.243804    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:36.260617 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260545    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.260798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260628    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.260798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260650    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.260798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260687    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.260798 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260777    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.260944 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.260822    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rppb\" (UniqueName: \"kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.361863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361821    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.361863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361864    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361882    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361910    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361934    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.361957    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rppb\" (UniqueName: \"kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362332 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.362307    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.362338    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.362395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.362348    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.364420 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.364389    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.364643 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.364623    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.370600 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.370577    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rppb\" (UniqueName: \"kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb\") pod \"rate-test-model-905fec-kserve-7f7857b8b7-bv7nz\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") " pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.459345 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.459262    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:36.739219 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:36.739193    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:55:36.741747 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:55:36.741717    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e7b2864_356f_49be_9c71_ce3928d1cb12.slice/crio-b58ab64a71494f889c6210dc61c858480543b99c91705b0ae71c334e926454fc WatchSource:0}: Error finding container b58ab64a71494f889c6210dc61c858480543b99c91705b0ae71c334e926454fc: Status 404 returned error can't find the container with id b58ab64a71494f889c6210dc61c858480543b99c91705b0ae71c334e926454fc
Sep 08 13:55:37.755341 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:37.755302    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" event={"ID":"9e7b2864-356f-49be-9c71-ce3928d1cb12","Type":"ContainerStarted","Data":"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"}
Sep 08 13:55:37.755800 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:37.755353    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" event={"ID":"9e7b2864-356f-49be-9c71-ce3928d1cb12","Type":"ContainerStarted","Data":"b58ab64a71494f889c6210dc61c858480543b99c91705b0ae71c334e926454fc"}
Sep 08 13:55:37.772946 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:37.772892    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" podStartSLOduration=1.7728752239999999 podStartE2EDuration="1.772875224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:55:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:55:37.771807218 +0000 UTC m=+1623.989723543" watchObservedRunningTime="2026-09-08 13:55:37.772875224 +0000 UTC m=+1623.990791549"
Sep 08 13:55:37.801505 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:37.801470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:37.901318 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:37.901279    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:38.100539 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:38.100498    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:38.240546 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:38.240514    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:55:38.683337 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:38.683289    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:46.459725 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:46.459686    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:46.460328 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:46.459746    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:46.472728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:46.472701    2583 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:46.826827 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:46.826799    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:55:51.152145 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:51.152108    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:55:54.054025 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.053990    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:55:54.054566 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.054250    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" podUID="772b736c-98bf-4983-91eb-7e76f0fffec8" containerName="maas-api" containerID="cri-o://5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd" gracePeriod=30
Sep 08 13:55:54.095378 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.095343    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:54.197038 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.197003    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:54.294849 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.294819    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:54.322053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.322026    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:55:54.352905 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.352875    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2x5mk\" (UniqueName: \"kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk\") pod \"772b736c-98bf-4983-91eb-7e76f0fffec8\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") "
Sep 08 13:55:54.353065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.352964    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs\") pod \"772b736c-98bf-4983-91eb-7e76f0fffec8\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") "
Sep 08 13:55:54.353065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.352993    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca\") pod \"772b736c-98bf-4983-91eb-7e76f0fffec8\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") "
Sep 08 13:55:54.353065 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.353027    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") pod \"772b736c-98bf-4983-91eb-7e76f0fffec8\" (UID: \"772b736c-98bf-4983-91eb-7e76f0fffec8\") "
Sep 08 13:55:54.353397 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.353375    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "772b736c-98bf-4983-91eb-7e76f0fffec8" (UID: "772b736c-98bf-4983-91eb-7e76f0fffec8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:55:54.355358 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.355328    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk" (OuterVolumeSpecName: "kube-api-access-2x5mk") pod "772b736c-98bf-4983-91eb-7e76f0fffec8" (UID: "772b736c-98bf-4983-91eb-7e76f0fffec8"). InnerVolumeSpecName "kube-api-access-2x5mk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:55:54.355455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.355323    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "772b736c-98bf-4983-91eb-7e76f0fffec8" (UID: "772b736c-98bf-4983-91eb-7e76f0fffec8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:55:54.355455 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.355407    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "772b736c-98bf-4983-91eb-7e76f0fffec8" (UID: "772b736c-98bf-4983-91eb-7e76f0fffec8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:55:54.394045 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.394008    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:54.454275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.454246    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2x5mk\" (UniqueName: \"kubernetes.io/projected/772b736c-98bf-4983-91eb-7e76f0fffec8-kube-api-access-2x5mk\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:54.454275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.454273    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:54.454275 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.454283    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/772b736c-98bf-4983-91eb-7e76f0fffec8-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:54.454543 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.454296    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/772b736c-98bf-4983-91eb-7e76f0fffec8-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:55:54.718303 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.718207    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:55:54.870527 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.870494    2583 generic.go:356] "Generic (PLEG): container finished" podID="772b736c-98bf-4983-91eb-7e76f0fffec8" containerID="5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd" exitCode=0
Sep 08 13:55:54.870724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.870536    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"
Sep 08 13:55:54.870724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.870582    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" event={"ID":"772b736c-98bf-4983-91eb-7e76f0fffec8","Type":"ContainerDied","Data":"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"}
Sep 08 13:55:54.870724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.870642    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849" event={"ID":"772b736c-98bf-4983-91eb-7e76f0fffec8","Type":"ContainerDied","Data":"fff9b944a9005ded5ff3660831bed5eab131e3c0e90cb9b338d3f63a9da323a1"}
Sep 08 13:55:54.870724 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.870665    2583 scope.go:117] "RemoveContainer" containerID="5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"
Sep 08 13:55:54.881333 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.881316    2583 scope.go:117] "RemoveContainer" containerID="5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"
Sep 08 13:55:54.881652 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:55:54.881620    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd\": container with ID starting with 5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd not found: ID does not exist" containerID="5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"
Sep 08 13:55:54.881755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.881661    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd"} err="failed to get container status \"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd\": rpc error: code = NotFound desc = could not find container \"5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd\": container with ID starting with 5c68211a9bcdea3cbba6a796ec048038c08cacc2274fdaae1e3525ff3ad6d3cd not found: ID does not exist"
Sep 08 13:55:54.895774 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.895744    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:55:54.900755 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:54.900731    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-8ef041-7df8d77c8-zk849"]
Sep 08 13:55:56.231961 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:56.231925    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="772b736c-98bf-4983-91eb-7e76f0fffec8" path="/var/lib/kubelet/pods/772b736c-98bf-4983-91eb-7e76f0fffec8/volumes"
Sep 08 13:55:58.068518 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:55:58.068475    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:00.862114 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:00.862078    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:00.998769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:00.998737    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:01.237826 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:01.237734    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:01.845544 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:01.845507    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:07.643370 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:07.643336    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:56:12.084229 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.084190    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:56:12.084788 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.084435    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" podUID="35d35d85-9bfc-4017-8c10-f484ebf9c401" containerName="maas-api" containerID="cri-o://0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7" gracePeriod=30
Sep 08 13:56:12.151927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.151884    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:12.305728 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.305686    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:12.369704 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.369680    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:56:12.392962 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.392930    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:12.449344 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449312    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs\") pod \"35d35d85-9bfc-4017-8c10-f484ebf9c401\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") "
Sep 08 13:56:12.449535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449375    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca\") pod \"35d35d85-9bfc-4017-8c10-f484ebf9c401\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") "
Sep 08 13:56:12.449535 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449505    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls\") pod \"35d35d85-9bfc-4017-8c10-f484ebf9c401\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") "
Sep 08 13:56:12.449697 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449621    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hh45m\" (UniqueName: \"kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m\") pod \"35d35d85-9bfc-4017-8c10-f484ebf9c401\" (UID: \"35d35d85-9bfc-4017-8c10-f484ebf9c401\") "
Sep 08 13:56:12.449822 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449763    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "35d35d85-9bfc-4017-8c10-f484ebf9c401" (UID: "35d35d85-9bfc-4017-8c10-f484ebf9c401"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:12.449999 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.449977    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/35d35d85-9bfc-4017-8c10-f484ebf9c401-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:12.451863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.451831    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "35d35d85-9bfc-4017-8c10-f484ebf9c401" (UID: "35d35d85-9bfc-4017-8c10-f484ebf9c401"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:56:12.451863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.451842    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "35d35d85-9bfc-4017-8c10-f484ebf9c401" (UID: "35d35d85-9bfc-4017-8c10-f484ebf9c401"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:56:12.452015 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.451990    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m" (OuterVolumeSpecName: "kube-api-access-hh45m") pod "35d35d85-9bfc-4017-8c10-f484ebf9c401" (UID: "35d35d85-9bfc-4017-8c10-f484ebf9c401"). InnerVolumeSpecName "kube-api-access-hh45m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:12.454119 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.454089    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:12.551129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.551080    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:12.551129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.551121    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/35d35d85-9bfc-4017-8c10-f484ebf9c401-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:12.551129 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.551131    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hh45m\" (UniqueName: \"kubernetes.io/projected/35d35d85-9bfc-4017-8c10-f484ebf9c401-kube-api-access-hh45m\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:12.843927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.843896    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:12.988451 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.988413    2583 generic.go:356] "Generic (PLEG): container finished" podID="35d35d85-9bfc-4017-8c10-f484ebf9c401" containerID="0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7" exitCode=0
Sep 08 13:56:12.988636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.988473    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"
Sep 08 13:56:12.988636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.988501    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" event={"ID":"35d35d85-9bfc-4017-8c10-f484ebf9c401","Type":"ContainerDied","Data":"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"}
Sep 08 13:56:12.988636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.988590    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws" event={"ID":"35d35d85-9bfc-4017-8c10-f484ebf9c401","Type":"ContainerDied","Data":"ff38aa17ed9bbdec3e3e544c1d37a7042f9c4b7bf73b413f365ca34120a82207"}
Sep 08 13:56:12.988636 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.988621    2583 scope.go:117] "RemoveContainer" containerID="0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"
Sep 08 13:56:12.999326 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.999299    2583 scope.go:117] "RemoveContainer" containerID="0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"
Sep 08 13:56:12.999630 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:56:12.999607    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7\": container with ID starting with 0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7 not found: ID does not exist" containerID="0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"
Sep 08 13:56:12.999718 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:12.999643    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7"} err="failed to get container status \"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7\": rpc error: code = NotFound desc = could not find container \"0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7\": container with ID starting with 0610e3ddb6c33a66c65d22ff563aa51b8dac6328d16366771f25518e36d4d8e7 not found: ID does not exist"
Sep 08 13:56:13.009703 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:13.009678    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:56:13.016227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:13.016202    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-3c9fc8-5d9b8c5494-whmws"]
Sep 08 13:56:14.232508 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:14.232473    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="35d35d85-9bfc-4017-8c10-f484ebf9c401" path="/var/lib/kubelet/pods/35d35d85-9bfc-4017-8c10-f484ebf9c401/volumes"
Sep 08 13:56:14.907812 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:14.907780    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:14.974860 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:14.974826    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:15.441604 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:15.441532    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:56:15.463708 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:15.459894    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:16.047011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:16.046976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:16.279180 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:16.279144    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:16.464958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:16.464858    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:18.845616 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:18.845573    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:18.919754 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:18.919707    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:19.147614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:19.147505    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:19.409997 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:19.409911    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:19.583471 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:19.583432    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:19.685967 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:19.685877    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:20.651017 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.650987    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:56:20.651440 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651427    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35d35d85-9bfc-4017-8c10-f484ebf9c401" containerName="maas-api"
Sep 08 13:56:20.651440 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651441    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="35d35d85-9bfc-4017-8c10-f484ebf9c401" containerName="maas-api"
Sep 08 13:56:20.651517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651458    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="772b736c-98bf-4983-91eb-7e76f0fffec8" containerName="maas-api"
Sep 08 13:56:20.651517 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651464    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="772b736c-98bf-4983-91eb-7e76f0fffec8" containerName="maas-api"
Sep 08 13:56:20.651607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651542    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="35d35d85-9bfc-4017-8c10-f484ebf9c401" containerName="maas-api"
Sep 08 13:56:20.651607 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.651571    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="772b736c-98bf-4983-91eb-7e76f0fffec8" containerName="maas-api"
Sep 08 13:56:20.655927 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.655901    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.658058 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.658036    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-a-2199d0-serving-cert\""
Sep 08 13:56:20.667463 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.667433    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:56:20.686338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.686307    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:20.786230 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.786199    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:20.840925 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.840879    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrng6\" (UniqueName: \"kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.841121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.840970    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.841121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.841013    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.841121 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.841066    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.905616 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.905522    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:20.942093 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.942056    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrng6\" (UniqueName: \"kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.942264 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.942118    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.942264 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.942155    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.942264 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.942174    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.942930 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.942907    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.944811 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.944793    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.944899 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.944883    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.950471 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.950451    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrng6\" (UniqueName: \"kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6\") pod \"maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:20.971418 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:20.971382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:21.255212 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:21.255120    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:56:21.260326 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:21.260282    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:21.262135 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:56:21.262095    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4c7e0d_d214_4475_b1e1_8e65f840c4c3.slice/crio-bfe54de510ce45fc599c4cc932b21cf2b741f9148f7bc0d04fab048ac5668d3f WatchSource:0}: Error finding container bfe54de510ce45fc599c4cc932b21cf2b741f9148f7bc0d04fab048ac5668d3f: Status 404 returned error can't find the container with id bfe54de510ce45fc599c4cc932b21cf2b741f9148f7bc0d04fab048ac5668d3f
Sep 08 13:56:21.263414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:21.263397    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 13:56:21.335958 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:21.335924    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:21.848379 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:21.848326    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:22.053414 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:22.053369    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" event={"ID":"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3","Type":"ContainerStarted","Data":"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"}
Sep 08 13:56:22.053627 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:22.053425    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" event={"ID":"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3","Type":"ContainerStarted","Data":"bfe54de510ce45fc599c4cc932b21cf2b741f9148f7bc0d04fab048ac5668d3f"}
Sep 08 13:56:22.053627 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:22.053519    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:22.073486 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:22.073433    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" podStartSLOduration=1.478337577 podStartE2EDuration="2.073414899s" totalImagesPullingTime="595.077322ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:22.072453415 +0000 UTC m=+1668.290369760" watchObservedRunningTime="2026-09-08 13:56:22.073414899 +0000 UTC m=+1668.291331227"
Sep 08 13:56:22.582080 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:22.582050    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:26.342042 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:26.342008    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:56:28.064688 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:28.064658    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:56:29.826168 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:29.826136    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:30.451217 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.451179    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:56:30.451511 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.451478    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" podUID="350be2e4-e363-46a2-b616-b1e92a1c3188" containerName="maas-api" containerID="cri-o://94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede" gracePeriod=30
Sep 08 13:56:30.469062 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.469022    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" podUID="9e7b2864-356f-49be-9c71-ce3928d1cb12" containerName="main" containerID="cri-o://5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68" gracePeriod=45
Sep 08 13:56:30.482076 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.482044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:30.567510 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.567477    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:30.668356 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.668309    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:30.730417 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.730380    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:30.737388 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.737366    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:56:30.766719 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.766695    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:56:30.834084 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834044    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-88t59\" (UniqueName: \"kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59\") pod \"350be2e4-e363-46a2-b616-b1e92a1c3188\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834111    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") pod \"350be2e4-e363-46a2-b616-b1e92a1c3188\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834163    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834226    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca\") pod \"350be2e4-e363-46a2-b616-b1e92a1c3188\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834275    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834312    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834339    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs\") pod \"350be2e4-e363-46a2-b616-b1e92a1c3188\" (UID: \"350be2e4-e363-46a2-b616-b1e92a1c3188\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834379    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834459    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rppb\" (UniqueName: \"kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.834522 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834493    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache\") pod \"9e7b2864-356f-49be-9c71-ce3928d1cb12\" (UID: \"9e7b2864-356f-49be-9c71-ce3928d1cb12\") "
Sep 08 13:56:30.835037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834780    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home" (OuterVolumeSpecName: "home") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:30.835037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834808    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "350be2e4-e363-46a2-b616-b1e92a1c3188" (UID: "350be2e4-e363-46a2-b616-b1e92a1c3188"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:56:30.835037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834952    2583 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-home\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.835037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.834971    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/350be2e4-e363-46a2-b616-b1e92a1c3188-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.835248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.835091    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:30.835248 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.835150    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache" (OuterVolumeSpecName: "model-cache") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:30.836775 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.836739    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:56:30.837092 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.837058    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm" (OuterVolumeSpecName: "dshm") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 13:56:30.837203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.837058    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "350be2e4-e363-46a2-b616-b1e92a1c3188" (UID: "350be2e4-e363-46a2-b616-b1e92a1c3188"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:56:30.837203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.837095    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59" (OuterVolumeSpecName: "kube-api-access-88t59") pod "350be2e4-e363-46a2-b616-b1e92a1c3188" (UID: "350be2e4-e363-46a2-b616-b1e92a1c3188"). InnerVolumeSpecName "kube-api-access-88t59". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:30.837203 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.837174    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb" (OuterVolumeSpecName: "kube-api-access-6rppb") pod "9e7b2864-356f-49be-9c71-ce3928d1cb12" (UID: "9e7b2864-356f-49be-9c71-ce3928d1cb12"). InnerVolumeSpecName "kube-api-access-6rppb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:56:30.837375 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.837358    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "350be2e4-e363-46a2-b616-b1e92a1c3188" (UID: "350be2e4-e363-46a2-b616-b1e92a1c3188"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:56:30.936228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936189    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-88t59\" (UniqueName: \"kubernetes.io/projected/350be2e4-e363-46a2-b616-b1e92a1c3188-kube-api-access-88t59\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936222    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936233    2583 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e7b2864-356f-49be-9c71-ce3928d1cb12-tls-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936228 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936240    2583 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-dshm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936248    2583 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-tmp-dir\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936256    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/350be2e4-e363-46a2-b616-b1e92a1c3188-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936263    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rppb\" (UniqueName: \"kubernetes.io/projected/9e7b2864-356f-49be-9c71-ce3928d1cb12-kube-api-access-6rppb\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:30.936516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:30.936271    2583 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e7b2864-356f-49be-9c71-ce3928d1cb12-model-cache\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:56:31.112975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.112941    2583 generic.go:356] "Generic (PLEG): container finished" podID="350be2e4-e363-46a2-b616-b1e92a1c3188" containerID="94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede" exitCode=0
Sep 08 13:56:31.112975 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.112982    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"
Sep 08 13:56:31.113205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.113016    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" event={"ID":"350be2e4-e363-46a2-b616-b1e92a1c3188","Type":"ContainerDied","Data":"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"}
Sep 08 13:56:31.113205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.113069    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48" event={"ID":"350be2e4-e363-46a2-b616-b1e92a1c3188","Type":"ContainerDied","Data":"2d3e2caedc8a91b18ff995a5fda9d5b07961a9cdae4c54b65cf3c5c1a3d59771"}
Sep 08 13:56:31.113205 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.113105    2583 scope.go:117] "RemoveContainer" containerID="94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"
Sep 08 13:56:31.114717 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.114692    2583 generic.go:356] "Generic (PLEG): container finished" podID="9e7b2864-356f-49be-9c71-ce3928d1cb12" containerID="5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68" exitCode=0
Sep 08 13:56:31.114829 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.114745    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" event={"ID":"9e7b2864-356f-49be-9c71-ce3928d1cb12","Type":"ContainerDied","Data":"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"}
Sep 08 13:56:31.114829 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.114770    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz" event={"ID":"9e7b2864-356f-49be-9c71-ce3928d1cb12","Type":"ContainerDied","Data":"b58ab64a71494f889c6210dc61c858480543b99c91705b0ae71c334e926454fc"}
Sep 08 13:56:31.114912 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.114880    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"
Sep 08 13:56:31.128937 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.128916    2583 scope.go:117] "RemoveContainer" containerID="94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"
Sep 08 13:56:31.129243 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:56:31.129225    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede\": container with ID starting with 94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede not found: ID does not exist" containerID="94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"
Sep 08 13:56:31.129308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.129250    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede"} err="failed to get container status \"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede\": rpc error: code = NotFound desc = could not find container \"94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede\": container with ID starting with 94107bdaccb89835014ebcfd50fe6c2711294e56abaea7eb2dc38199ffd0eede not found: ID does not exist"
Sep 08 13:56:31.129308 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.129266    2583 scope.go:117] "RemoveContainer" containerID="5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"
Sep 08 13:56:31.138027 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.138008    2583 scope.go:117] "RemoveContainer" containerID="5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"
Sep 08 13:56:31.138285 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:56:31.138268    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68\": container with ID starting with 5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68 not found: ID does not exist" containerID="5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"
Sep 08 13:56:31.138353 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.138292    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68"} err="failed to get container status \"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68\": rpc error: code = NotFound desc = could not find container \"5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68\": container with ID starting with 5114417586d49c6af50d651c48e5376550ef04d703f8ec1bf4f483611fd39d68 not found: ID does not exist"
Sep 08 13:56:31.139172 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.139151    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:56:31.142999 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.142974    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-d9970e-6746d99c5-qch48"]
Sep 08 13:56:31.153493 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.153467    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:56:31.157652 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.157625    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-905fec/rate-test-model-905fec-kserve-7f7857b8b7-bv7nz"]
Sep 08 13:56:31.163053 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:31.163030    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:32.231364 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:32.231332    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="350be2e4-e363-46a2-b616-b1e92a1c3188" path="/var/lib/kubelet/pods/350be2e4-e363-46a2-b616-b1e92a1c3188/volumes"
Sep 08 13:56:32.231758 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:32.231724    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e7b2864-356f-49be-9c71-ce3928d1cb12" path="/var/lib/kubelet/pods/9e7b2864-356f-49be-9c71-ce3928d1cb12/volumes"
Sep 08 13:56:36.999399 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:36.999364    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:37.100891 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:37.100862    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:37.343079 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:37.343052    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:40.032160 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:40.032117    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:56:41.789690 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:41.789651    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:44.513572 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:44.513519    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:45.796567 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:45.796523    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:45.898582 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:45.898515    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:46.103840 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:46.103743    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:48.979040 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:48.978995    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:49.215612 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:49.215542    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.072655 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.072610    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 13:56:50.073315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073297    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e7b2864-356f-49be-9c71-ce3928d1cb12" containerName="main"
Sep 08 13:56:50.073389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073317    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e7b2864-356f-49be-9c71-ce3928d1cb12" containerName="main"
Sep 08 13:56:50.073389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073366    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="350be2e4-e363-46a2-b616-b1e92a1c3188" containerName="maas-api"
Sep 08 13:56:50.073389 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073375    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="350be2e4-e363-46a2-b616-b1e92a1c3188" containerName="maas-api"
Sep 08 13:56:50.073540 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073472    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="350be2e4-e363-46a2-b616-b1e92a1c3188" containerName="maas-api"
Sep 08 13:56:50.073540 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.073487    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e7b2864-356f-49be-9c71-ce3928d1cb12" containerName="main"
Sep 08 13:56:50.078863 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.078837    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.081280 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.081251    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-cdf4f2-serving-cert\""
Sep 08 13:56:50.088626 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.088173    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 13:56:50.108531 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.108497    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.233420 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.233390    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.233646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.233472    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qfvdd\" (UniqueName: \"kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.233646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.233535    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.233646 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.233578    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.334793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.334707    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qfvdd\" (UniqueName: \"kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.334793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.334779    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.334993 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.334801    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.334993 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.334831    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.335624 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.335531    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.337681 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.337485    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.341589 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.337957    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.345614 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.344726    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qfvdd\" (UniqueName: \"kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd\") pod \"maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.384893 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.384852    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.394751 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.394722    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:50.485395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.485361    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.599261 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.599167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.676426 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.676393    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 13:56:50.677595 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:50.677489    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:56:50.679302 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:56:50.679277    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ab7a189_0f00_4e69_8c5f_986a72e2ab8d.slice/crio-0e839a2fdae908087fa1fa53ea342a2b6b131e7d7ee7cd77ade29dba97c30bee WatchSource:0}: Error finding container 0e839a2fdae908087fa1fa53ea342a2b6b131e7d7ee7cd77ade29dba97c30bee: Status 404 returned error can't find the container with id 0e839a2fdae908087fa1fa53ea342a2b6b131e7d7ee7cd77ade29dba97c30bee
Sep 08 13:56:51.241565 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:51.241522    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" event={"ID":"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d","Type":"ContainerStarted","Data":"0e839a2fdae908087fa1fa53ea342a2b6b131e7d7ee7cd77ade29dba97c30bee"}
Sep 08 13:56:52.248761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:52.248721    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" event={"ID":"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d","Type":"ContainerStarted","Data":"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"}
Sep 08 13:56:52.249220 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:52.248842    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:56:52.270191 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:52.270128    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" podStartSLOduration=1.441160366 podStartE2EDuration="2.270111787s" totalImagesPullingTime="828.951421ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:56:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:56:52.26711892 +0000 UTC m=+1698.485035290" watchObservedRunningTime="2026-09-08 13:56:52.270111787 +0000 UTC m=+1698.488028113"
Sep 08 13:56:54.847468 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:54.847430    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:56:58.261363 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:56:58.261285    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 13:57:05.066044 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:05.066006    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:05.130479 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:05.130446    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:13.938793 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:13.938757    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:29.178799 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:29.178763    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:33.039176 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:33.039143    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:57:34.851961 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:34.851927    2583 scope.go:117] "RemoveContainer" containerID="e8555730cdf7d94c45806e1878ab8f2d6e9b04a0eef5094f28460907b603ead1"
Sep 08 13:57:34.861505 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:34.861484    2583 scope.go:117] "RemoveContainer" containerID="83fbf06640cdc896e479649d54de36bc23f75b48d58384b532f0159a5430f4f1"
Sep 08 13:57:43.609637 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:43.609596    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:43.667222 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:43.667191    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:51.532150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:51.532112    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:57:55.634097 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:55.634061    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:55.967374 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:55.967285    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:56.382706 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:56.382653    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:58.897900 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:58.897863    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:57:58.996663 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:57:58.996625    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:00.133146 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.133097    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:00.137324 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.137295    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.139726 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.139676    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-c780a1-serving-cert\""
Sep 08 13:58:00.148885 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.148858    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:00.246908 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.246868    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.247107 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.247008    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.247107 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.247038    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnqtq\" (UniqueName: \"kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.247223 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.247132    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.341903 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.341870    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:00.347973 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.347939    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.348110 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.348059    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.348279 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.348259    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.348429 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.348409    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnqtq\" (UniqueName: \"kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.348877 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.348824    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.350687 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.350665    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.350759 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.350716    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.357037 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.357018    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnqtq\" (UniqueName: \"kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq\") pod \"maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.439189 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.439091    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:00.450469 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.450442    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:00.643169 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.643131    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:00.738118 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.738084    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:00.739251 ip-10-0-140-23 kubenswrapper[2583]: W0908 13:58:00.739224    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d913225_b28a_4f23_92d6_46d89f241fd2.slice/crio-b476fe1f5ea9443591c3a485b47674e281546e924b6e5562f1e28c59bb967399 WatchSource:0}: Error finding container b476fe1f5ea9443591c3a485b47674e281546e924b6e5562f1e28c59bb967399: Status 404 returned error can't find the container with id b476fe1f5ea9443591c3a485b47674e281546e924b6e5562f1e28c59bb967399
Sep 08 13:58:00.743359 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:00.743333    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:01.684490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:01.684405    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" event={"ID":"3d913225-b28a-4f23-92d6-46d89f241fd2","Type":"ContainerStarted","Data":"0839b3cafd7011f135458e2ff05f82c6e3ceba953346bad8f9f984f50d74a961"}
Sep 08 13:58:01.684490 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:01.684469    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" event={"ID":"3d913225-b28a-4f23-92d6-46d89f241fd2","Type":"ContainerStarted","Data":"b476fe1f5ea9443591c3a485b47674e281546e924b6e5562f1e28c59bb967399"}
Sep 08 13:58:01.684933 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:01.684641    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:01.702256 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:01.702201    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" podStartSLOduration=1.146428481 podStartE2EDuration="1.702180321s" totalImagesPullingTime="555.75184ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 13:58:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 13:58:01.7003568 +0000 UTC m=+1767.918273164" watchObservedRunningTime="2026-09-08 13:58:01.702180321 +0000 UTC m=+1767.920096648"
Sep 08 13:58:03.737071 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:03.737037    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:58:07.697432 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:07.697402    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:15.849594 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:15.849531    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:23.889919 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:23.889875    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:30.673410 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.673317    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:30.674282 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.674217    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" podUID="3d913225-b28a-4f23-92d6-46d89f241fd2" containerName="maas-api" containerID="cri-o://0839b3cafd7011f135458e2ff05f82c6e3ceba953346bad8f9f984f50d74a961" gracePeriod=30
Sep 08 13:58:30.697995 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.697960    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:30.800302 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.800260    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:30.868492 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.868456    2583 generic.go:356] "Generic (PLEG): container finished" podID="3d913225-b28a-4f23-92d6-46d89f241fd2" containerID="0839b3cafd7011f135458e2ff05f82c6e3ceba953346bad8f9f984f50d74a961" exitCode=0
Sep 08 13:58:30.868699 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.868578    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" event={"ID":"3d913225-b28a-4f23-92d6-46d89f241fd2","Type":"ContainerDied","Data":"0839b3cafd7011f135458e2ff05f82c6e3ceba953346bad8f9f984f50d74a961"}
Sep 08 13:58:30.879113 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.879044    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:30.938501 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.938426    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:30.940081 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:30.940046    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:31.069165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069131    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca\") pod \"3d913225-b28a-4f23-92d6-46d89f241fd2\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") "
Sep 08 13:58:31.069165 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069172    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls\") pod \"3d913225-b28a-4f23-92d6-46d89f241fd2\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") "
Sep 08 13:58:31.069447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069233    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs\") pod \"3d913225-b28a-4f23-92d6-46d89f241fd2\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") "
Sep 08 13:58:31.069447 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069264    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnqtq\" (UniqueName: \"kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq\") pod \"3d913225-b28a-4f23-92d6-46d89f241fd2\" (UID: \"3d913225-b28a-4f23-92d6-46d89f241fd2\") "
Sep 08 13:58:31.069656 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069630    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3d913225-b28a-4f23-92d6-46d89f241fd2" (UID: "3d913225-b28a-4f23-92d6-46d89f241fd2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:31.069805 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.069789    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3d913225-b28a-4f23-92d6-46d89f241fd2-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.071669 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.071635    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "3d913225-b28a-4f23-92d6-46d89f241fd2" (UID: "3d913225-b28a-4f23-92d6-46d89f241fd2"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:58:31.071766 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.071668    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3d913225-b28a-4f23-92d6-46d89f241fd2" (UID: "3d913225-b28a-4f23-92d6-46d89f241fd2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:58:31.071766 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.071743    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq" (OuterVolumeSpecName: "kube-api-access-vnqtq") pod "3d913225-b28a-4f23-92d6-46d89f241fd2" (UID: "3d913225-b28a-4f23-92d6-46d89f241fd2"). InnerVolumeSpecName "kube-api-access-vnqtq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:31.171024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.170986    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.171024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.171018    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3d913225-b28a-4f23-92d6-46d89f241fd2-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.171024 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.171027    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnqtq\" (UniqueName: \"kubernetes.io/projected/3d913225-b28a-4f23-92d6-46d89f241fd2-kube-api-access-vnqtq\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:31.260971 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.260861    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:31.876634 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.876603    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"
Sep 08 13:58:31.876634 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.876619    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f" event={"ID":"3d913225-b28a-4f23-92d6-46d89f241fd2","Type":"ContainerDied","Data":"b476fe1f5ea9443591c3a485b47674e281546e924b6e5562f1e28c59bb967399"}
Sep 08 13:58:31.877150 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.876692    2583 scope.go:117] "RemoveContainer" containerID="0839b3cafd7011f135458e2ff05f82c6e3ceba953346bad8f9f984f50d74a961"
Sep 08 13:58:31.901395 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.901360    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:31.907504 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:31.907477    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-c780a1-668c776885-zcr4f"]
Sep 08 13:58:32.232239 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:32.232154    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d913225-b28a-4f23-92d6-46d89f241fd2" path="/var/lib/kubelet/pods/3d913225-b28a-4f23-92d6-46d89f241fd2/volumes"
Sep 08 13:58:34.381449 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:34.381420    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:58:34.387678 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:34.387648    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 13:58:36.942769 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:36.942729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:58:37.103747 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:37.103704    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:37.166515 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:37.166480    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:37.370315 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:37.370263    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:39.162870 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:39.162833    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:41.188856 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:41.188824    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:56.584516 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.584475    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:58:56.585011 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.584800    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" podUID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" containerName="maas-api" containerID="cri-o://053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e" gracePeriod=30
Sep 08 13:58:56.614527 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.614499    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:56.690687 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.690654    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:56.785338 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.785273    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:56.849623 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.849540    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:56.850481 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.850464    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:58:56.921402 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.921365    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls\") pod \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") "
Sep 08 13:58:56.921593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.921415    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nrng6\" (UniqueName: \"kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6\") pod \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") "
Sep 08 13:58:56.921593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.921467    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs\") pod \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") "
Sep 08 13:58:56.921593 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.921540    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca\") pod \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\" (UID: \"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3\") "
Sep 08 13:58:56.921992 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.921961    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" (UID: "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 13:58:56.923751 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.923726    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6" (OuterVolumeSpecName: "kube-api-access-nrng6") pod "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" (UID: "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3"). InnerVolumeSpecName "kube-api-access-nrng6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 13:58:56.923841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.923768    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" (UID: "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:58:56.923841 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:56.923828    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" (UID: "fc4c7e0d-d214-4475-b1e1-8e65f840c4c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 13:58:57.023485 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.023446    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.023485 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.023480    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.023485 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.023491    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.023761 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.023501    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrng6\" (UniqueName: \"kubernetes.io/projected/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3-kube-api-access-nrng6\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 13:58:57.035938 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.035906    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" containerID="053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e" exitCode=0
Sep 08 13:58:57.036099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.035947    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" event={"ID":"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3","Type":"ContainerDied","Data":"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"}
Sep 08 13:58:57.036099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.036002    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp" event={"ID":"fc4c7e0d-d214-4475-b1e1-8e65f840c4c3","Type":"ContainerDied","Data":"bfe54de510ce45fc599c4cc932b21cf2b741f9148f7bc0d04fab048ac5668d3f"}
Sep 08 13:58:57.036099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.036024    2583 scope.go:117] "RemoveContainer" containerID="053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"
Sep 08 13:58:57.036099 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.035956    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"
Sep 08 13:58:57.047159 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.047131    2583 scope.go:117] "RemoveContainer" containerID="053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"
Sep 08 13:58:57.047491 ip-10-0-140-23 kubenswrapper[2583]: E0908 13:58:57.047468    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e\": container with ID starting with 053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e not found: ID does not exist" containerID="053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"
Sep 08 13:58:57.047609 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.047499    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e"} err="failed to get container status \"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e\": rpc error: code = NotFound desc = could not find container \"053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e\": container with ID starting with 053f20797557e6a4b63cb0767a6ec6331b6ad8b6aa4e74c6c5b0fd9255f9352e not found: ID does not exist"
Sep 08 13:58:57.057907 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.057883    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:58:57.064710 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.063606    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-a-2199d0-ff4bf8c47-sh7hp"]
Sep 08 13:58:57.205266 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.205176    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:58:57.836738 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:57.836703    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:58:58.231654 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:58:58.231540    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" path="/var/lib/kubelet/pods/fc4c7e0d-d214-4475-b1e1-8e65f840c4c3/volumes"
Sep 08 13:59:02.281771 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:02.281708    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:59:07.434245 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:07.434204    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 13:59:38.521872 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:38.521839    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:59:39.707307 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:39.707269    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:59:44.849227 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:44.849186    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:59:52.683255 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:52.683223    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 13:59:59.380196 ip-10-0-140-23 kubenswrapper[2583]: I0908 13:59:59.380098    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:00.330805 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:00.330766    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:00.434393 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:00.434351    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:10.582384 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.582344    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 14:00:10.582949 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.582636    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" podUID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" containerName="maas-api" containerID="cri-o://59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9" gracePeriod=30
Sep 08 14:00:10.607846 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.607804    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:10.849301 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.849230    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 14:00:10.983512 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.983464    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls\") pod \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") "
Sep 08 14:00:10.983512 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.983518    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca\") pod \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") "
Sep 08 14:00:10.983795 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.983560    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qfvdd\" (UniqueName: \"kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd\") pod \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") "
Sep 08 14:00:10.983795 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.983602    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs\") pod \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\" (UID: \"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d\") "
Sep 08 14:00:10.983934 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.983909    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" (UID: "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:10.984037 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.984023    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:10.986006 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.985976    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd" (OuterVolumeSpecName: "kube-api-access-qfvdd") pod "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" (UID: "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d"). InnerVolumeSpecName "kube-api-access-qfvdd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:10.986006 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.985990    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" (UID: "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:10.986162 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:10.985976    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" (UID: "9ab7a189-0f00-4e69-8c5f-986a72e2ab8d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:11.084939 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.084898    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:11.084939 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.084933    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qfvdd\" (UniqueName: \"kubernetes.io/projected/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-kube-api-access-qfvdd\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:11.084939 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.084943    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:11.513429 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.513389    2583 generic.go:356] "Generic (PLEG): container finished" podID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" containerID="59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9" exitCode=0
Sep 08 14:00:11.513658 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.513444    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"
Sep 08 14:00:11.513658 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.513475    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" event={"ID":"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d","Type":"ContainerDied","Data":"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"}
Sep 08 14:00:11.513658 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.513533    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9" event={"ID":"9ab7a189-0f00-4e69-8c5f-986a72e2ab8d","Type":"ContainerDied","Data":"0e839a2fdae908087fa1fa53ea342a2b6b131e7d7ee7cd77ade29dba97c30bee"}
Sep 08 14:00:11.513658 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.513589    2583 scope.go:117] "RemoveContainer" containerID="59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"
Sep 08 14:00:11.526786 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.526722    2583 scope.go:117] "RemoveContainer" containerID="59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"
Sep 08 14:00:11.528375 ip-10-0-140-23 kubenswrapper[2583]: E0908 14:00:11.528339    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9\": container with ID starting with 59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9 not found: ID does not exist" containerID="59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"
Sep 08 14:00:11.528499 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.528384    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9"} err="failed to get container status \"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9\": rpc error: code = NotFound desc = could not find container \"59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9\": container with ID starting with 59990d4b607fa7a6908a086a38f56257ad0d955a6e4a45ccc3580319095e13a9 not found: ID does not exist"
Sep 08 14:00:11.538120 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.538089    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 14:00:11.542391 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.542357    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-cdf4f2-7f6cbbbd64-xj6g9"]
Sep 08 14:00:11.570565 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.570511    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:11.632450 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:11.632415    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:12.232562 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:12.232513    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" path="/var/lib/kubelet/pods/9ab7a189-0f00-4e69-8c5f-986a72e2ab8d/volumes"
Sep 08 14:00:14.643997 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:14.643961    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:00:17.031138 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:17.031081    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:17.090915 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:17.090876    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:17.330125 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:17.330035    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:17.957091 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:17.957054    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:22.792298 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:22.792265    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:27.843571 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:27.843513    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:28.377030 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.376983    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 14:00:28.377325 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.377272    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" podUID="a1d828dc-be7c-463d-b488-42650a5d5b51" containerName="maas-api" containerID="cri-o://e8e35792e26b2daed1174e75cb94f592d011660ef5304dd8f62ed9e490d51af1" gracePeriod=30
Sep 08 14:00:28.410280 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.410245    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:28.509723 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.509648    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:28.604506 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.604470    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:28.624528 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.624494    2583 generic.go:356] "Generic (PLEG): container finished" podID="a1d828dc-be7c-463d-b488-42650a5d5b51" containerID="e8e35792e26b2daed1174e75cb94f592d011660ef5304dd8f62ed9e490d51af1" exitCode=0
Sep 08 14:00:28.624690 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.624572    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" event={"ID":"a1d828dc-be7c-463d-b488-42650a5d5b51","Type":"ContainerDied","Data":"e8e35792e26b2daed1174e75cb94f592d011660ef5304dd8f62ed9e490d51af1"}
Sep 08 14:00:28.644229 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.644198    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 14:00:28.656598 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.656567    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vt5jl\" (UniqueName: \"kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl\") pod \"a1d828dc-be7c-463d-b488-42650a5d5b51\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") "
Sep 08 14:00:28.656792 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.656608    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") pod \"a1d828dc-be7c-463d-b488-42650a5d5b51\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") "
Sep 08 14:00:28.656792 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.656680    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs\") pod \"a1d828dc-be7c-463d-b488-42650a5d5b51\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") "
Sep 08 14:00:28.656792 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.656779    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca\") pod \"a1d828dc-be7c-463d-b488-42650a5d5b51\" (UID: \"a1d828dc-be7c-463d-b488-42650a5d5b51\") "
Sep 08 14:00:28.657460 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.657363    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a1d828dc-be7c-463d-b488-42650a5d5b51" (UID: "a1d828dc-be7c-463d-b488-42650a5d5b51"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:28.659025 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.658991    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a1d828dc-be7c-463d-b488-42650a5d5b51" (UID: "a1d828dc-be7c-463d-b488-42650a5d5b51"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:28.659135 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.659066    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a1d828dc-be7c-463d-b488-42650a5d5b51" (UID: "a1d828dc-be7c-463d-b488-42650a5d5b51"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:28.659260 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.659239    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl" (OuterVolumeSpecName: "kube-api-access-vt5jl") pod "a1d828dc-be7c-463d-b488-42650a5d5b51" (UID: "a1d828dc-be7c-463d-b488-42650a5d5b51"). InnerVolumeSpecName "kube-api-access-vt5jl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:28.670190 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.670149    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:28.757992 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.757951    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt5jl\" (UniqueName: \"kubernetes.io/projected/a1d828dc-be7c-463d-b488-42650a5d5b51-kube-api-access-vt5jl\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.757992 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.757988    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.757992 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.758000    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a1d828dc-be7c-463d-b488-42650a5d5b51-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:28.758248 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:28.758011    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a1d828dc-be7c-463d-b488-42650a5d5b51-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:29.059650 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.059620    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:29.633116 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.633082    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"
Sep 08 14:00:29.633329 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.633128    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq" event={"ID":"a1d828dc-be7c-463d-b488-42650a5d5b51","Type":"ContainerDied","Data":"ae38b3ef1be078866cd6913efe9bf55ea0259130a4a0376baa5483e4bc0e8661"}
Sep 08 14:00:29.633329 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.633195    2583 scope.go:117] "RemoveContainer" containerID="e8e35792e26b2daed1174e75cb94f592d011660ef5304dd8f62ed9e490d51af1"
Sep 08 14:00:29.659460 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.659425    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 14:00:29.663252 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:29.663224    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-1f082e-7b8457964f-ncbkq"]
Sep 08 14:00:30.232149 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:30.232114    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a1d828dc-be7c-463d-b488-42650a5d5b51" path="/var/lib/kubelet/pods/a1d828dc-be7c-463d-b488-42650a5d5b51/volumes"
Sep 08 14:00:31.644839 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:31.644797    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:00:34.584875 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:34.584834    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:35.980961 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:35.980921    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:36.082951 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:36.082916    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.020582 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.020518    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 14:00:46.021138 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.020834    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" podUID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" containerName="maas-api" containerID="cri-o://84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483" gracePeriod=30
Sep 08 14:00:46.052870 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.052835    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.160020 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.159982    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.280998 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.280921    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 14:00:46.330214 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.330183    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca\") pod \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") "
Sep 08 14:00:46.330428 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.330225    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps2fk\" (UniqueName: \"kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk\") pod \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") "
Sep 08 14:00:46.330428 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.330244    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") pod \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") "
Sep 08 14:00:46.330428 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.330314    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs\") pod \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\" (UID: \"b5d3838f-671a-4eeb-a6ee-8c53744ae353\") "
Sep 08 14:00:46.331410 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.331373    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5d3838f-671a-4eeb-a6ee-8c53744ae353" (UID: "b5d3838f-671a-4eeb-a6ee-8c53744ae353"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:00:46.332834 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.332796    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk" (OuterVolumeSpecName: "kube-api-access-ps2fk") pod "b5d3838f-671a-4eeb-a6ee-8c53744ae353" (UID: "b5d3838f-671a-4eeb-a6ee-8c53744ae353"). InnerVolumeSpecName "kube-api-access-ps2fk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:00:46.333041 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.333012    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5d3838f-671a-4eeb-a6ee-8c53744ae353" (UID: "b5d3838f-671a-4eeb-a6ee-8c53744ae353"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:46.333041 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.333024    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b5d3838f-671a-4eeb-a6ee-8c53744ae353" (UID: "b5d3838f-671a-4eeb-a6ee-8c53744ae353"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:00:46.333284 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.333259    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.409875 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.409827    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.431320 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.431279    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.431320 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.431314    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d3838f-671a-4eeb-a6ee-8c53744ae353-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.431528 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.431327    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps2fk\" (UniqueName: \"kubernetes.io/projected/b5d3838f-671a-4eeb-a6ee-8c53744ae353-kube-api-access-ps2fk\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.431528 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.431340    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d3838f-671a-4eeb-a6ee-8c53744ae353-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:00:46.740187 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.740085    2583 generic.go:356] "Generic (PLEG): container finished" podID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" containerID="84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483" exitCode=0
Sep 08 14:00:46.740187 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.740133    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"
Sep 08 14:00:46.740187 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.740157    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" event={"ID":"b5d3838f-671a-4eeb-a6ee-8c53744ae353","Type":"ContainerDied","Data":"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"}
Sep 08 14:00:46.740431 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.740220    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9" event={"ID":"b5d3838f-671a-4eeb-a6ee-8c53744ae353","Type":"ContainerDied","Data":"481dc998b0811aa2f3fe9bcdc8e5e8200f1a4b84b9c7ebec0694e1a2b14ffa48"}
Sep 08 14:00:46.740431 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.740242    2583 scope.go:117] "RemoveContainer" containerID="84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"
Sep 08 14:00:46.743364 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.743338    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:46.750497 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.750476    2583 scope.go:117] "RemoveContainer" containerID="84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"
Sep 08 14:00:46.750788 ip-10-0-140-23 kubenswrapper[2583]: E0908 14:00:46.750768    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483\": container with ID starting with 84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483 not found: ID does not exist" containerID="84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"
Sep 08 14:00:46.750845 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.750798    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483"} err="failed to get container status \"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483\": rpc error: code = NotFound desc = could not find container \"84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483\": container with ID starting with 84c4d74cc2a5c019765c974438221f6b1e76e01c707d4d6c474c28843e31a483 not found: ID does not exist"
Sep 08 14:00:46.763227 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.763197    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 14:00:46.765715 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:46.765689    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-905fec-67f6bd7f8b-c8qr9"]
Sep 08 14:00:48.231442 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:48.231405    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" path="/var/lib/kubelet/pods/b5d3838f-671a-4eeb-a6ee-8c53744ae353/volumes"
Sep 08 14:00:49.836876 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:49.836835    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:00:52.736097 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:52.736060    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:52.803759 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:52.803717    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:54.239421 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:54.239381    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:00:57.141676 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:00:57.141633    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:01:34.945789 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:01:34.945684    2583 scope.go:117] "RemoveContainer" containerID="09957b1a05d88554ac80a1321432497de27165326a3c24005023d3abec3525bb"
Sep 08 14:01:43.431702 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:01:43.431664    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:01:43.492649 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:01:43.492612    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:02:18.380898 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:18.380842    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:02:19.442948 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:19.442902    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:02:21.463397 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:21.463362    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:02:22.070066 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:22.070031    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:02:46.241634 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:46.241598    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:02:53.443898 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:02:53.443857    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:00.837754 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:00.837650    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:10.345837 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:10.345797    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:18.855649 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:18.855601    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:28.031634 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:28.031591    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:34.426944 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:34.426913    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 14:03:34.435520 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:34.435492    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9fth2_5e22f156-742a-4317-aafc-f354d6f9a56c/ovn-acl-logging/0.log"
Sep 08 14:03:37.836327 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:37.836286    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:03:45.024901 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.024864    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 14:03:45.025388 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.025063    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" podUID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" containerName="manager" containerID="cri-o://e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c" gracePeriod=10
Sep 08 14:03:45.668318 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.668288    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 14:03:45.779467 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.779429    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtpzm\" (UniqueName: \"kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm\") pod \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") "
Sep 08 14:03:45.779699 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.779510    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume\") pod \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\" (UID: \"05aabb52-6e5d-4d50-bb5a-e04638f96dee\") "
Sep 08 14:03:45.779872 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.779847    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "05aabb52-6e5d-4d50-bb5a-e04638f96dee" (UID: "05aabb52-6e5d-4d50-bb5a-e04638f96dee"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 08 14:03:45.781826 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.781801    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm" (OuterVolumeSpecName: "kube-api-access-gtpzm") pod "05aabb52-6e5d-4d50-bb5a-e04638f96dee" (UID: "05aabb52-6e5d-4d50-bb5a-e04638f96dee"). InnerVolumeSpecName "kube-api-access-gtpzm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:03:45.812171 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.812137    2583 generic.go:356] "Generic (PLEG): container finished" podID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" containerID="e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c" exitCode=0
Sep 08 14:03:45.812335 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.812185    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"
Sep 08 14:03:45.812335 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.812220    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" event={"ID":"05aabb52-6e5d-4d50-bb5a-e04638f96dee","Type":"ContainerDied","Data":"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"}
Sep 08 14:03:45.812335 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.812283    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv" event={"ID":"05aabb52-6e5d-4d50-bb5a-e04638f96dee","Type":"ContainerDied","Data":"2b5d6d71ee70eccb462e9d95aeb427352d8d89ec6fbaefb13c7067cecc6f2e36"}
Sep 08 14:03:45.812335 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.812312    2583 scope.go:117] "RemoveContainer" containerID="e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"
Sep 08 14:03:45.823971 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.823949    2583 scope.go:117] "RemoveContainer" containerID="e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"
Sep 08 14:03:45.824302 ip-10-0-140-23 kubenswrapper[2583]: E0908 14:03:45.824281    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c\": container with ID starting with e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c not found: ID does not exist" containerID="e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"
Sep 08 14:03:45.824357 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.824312    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c"} err="failed to get container status \"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c\": rpc error: code = NotFound desc = could not find container \"e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c\": container with ID starting with e2bb17829d8e8464ea403de87bc26a12857323f6a8823beadc9a46523017522c not found: ID does not exist"
Sep 08 14:03:45.836628 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.836599    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 14:03:45.840755 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.840725    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-vp2gv"]
Sep 08 14:03:45.880715 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.880677    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtpzm\" (UniqueName: \"kubernetes.io/projected/05aabb52-6e5d-4d50-bb5a-e04638f96dee-kube-api-access-gtpzm\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:03:45.880715 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:45.880712    2583 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/05aabb52-6e5d-4d50-bb5a-e04638f96dee-extensions-socket-volume\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:03:46.231612 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:03:46.231507    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" path="/var/lib/kubelet/pods/05aabb52-6e5d-4d50-bb5a-e04638f96dee/volumes"
Sep 08 14:04:45.717790 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.717750    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"]
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718194    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718205    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718220    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" containerName="manager"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718226    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" containerName="manager"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718232    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718238    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718250    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d913225-b28a-4f23-92d6-46d89f241fd2" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718255    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d913225-b28a-4f23-92d6-46d89f241fd2" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718264    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a1d828dc-be7c-463d-b488-42650a5d5b51" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718269    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="a1d828dc-be7c-463d-b488-42650a5d5b51" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718277    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" containerName="maas-api"
Sep 08 14:04:45.718282 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718282    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" containerName="maas-api"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718343    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="a1d828dc-be7c-463d-b488-42650a5d5b51" containerName="maas-api"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718351    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d913225-b28a-4f23-92d6-46d89f241fd2" containerName="maas-api"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718357    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="05aabb52-6e5d-4d50-bb5a-e04638f96dee" containerName="manager"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718365    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4c7e0d-d214-4475-b1e1-8e65f840c4c3" containerName="maas-api"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718371    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab7a189-0f00-4e69-8c5f-986a72e2ab8d" containerName="maas-api"
Sep 08 14:04:45.718683 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.718378    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d3838f-671a-4eeb-a6ee-8c53744ae353" containerName="maas-api"
Sep 08 14:04:45.721629 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.721611    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.724278 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.724254    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-rdd2z\""
Sep 08 14:04:45.732320 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.732291    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"]
Sep 08 14:04:45.754916 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.754873    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71e22ddc-5cf5-464a-982a-afc300cf8206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.755093 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.754959    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rb4nl\" (UniqueName: \"kubernetes.io/projected/71e22ddc-5cf5-464a-982a-afc300cf8206-kube-api-access-rb4nl\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.856483 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.856440    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71e22ddc-5cf5-464a-982a-afc300cf8206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.856702 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.856507    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rb4nl\" (UniqueName: \"kubernetes.io/projected/71e22ddc-5cf5-464a-982a-afc300cf8206-kube-api-access-rb4nl\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.856919 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.856898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/71e22ddc-5cf5-464a-982a-afc300cf8206-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:45.864931 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:45.864898    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rb4nl\" (UniqueName: \"kubernetes.io/projected/71e22ddc-5cf5-464a-982a-afc300cf8206-kube-api-access-rb4nl\") pod \"kuadrant-operator-controller-manager-5b65f94d4c-8pc6m\" (UID: \"71e22ddc-5cf5-464a-982a-afc300cf8206\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:46.033247 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:46.033207    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:46.318071 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:46.318042    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"]
Sep 08 14:04:46.319471 ip-10-0-140-23 kubenswrapper[2583]: W0908 14:04:46.319442    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71e22ddc_5cf5_464a_982a_afc300cf8206.slice/crio-373759900c91f7aa345d62182eb7b495c6f1a5325d182e7a4257758876fe3e3a WatchSource:0}: Error finding container 373759900c91f7aa345d62182eb7b495c6f1a5325d182e7a4257758876fe3e3a: Status 404 returned error can't find the container with id 373759900c91f7aa345d62182eb7b495c6f1a5325d182e7a4257758876fe3e3a
Sep 08 14:04:47.186308 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:47.186265    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m" event={"ID":"71e22ddc-5cf5-464a-982a-afc300cf8206","Type":"ContainerStarted","Data":"d1f91074cb3792cdb7b6a9815cd151c7435825a89dece473084702e47ae6f63d"}
Sep 08 14:04:47.186753 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:47.186325    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:04:47.186753 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:47.186348    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m" event={"ID":"71e22ddc-5cf5-464a-982a-afc300cf8206","Type":"ContainerStarted","Data":"373759900c91f7aa345d62182eb7b495c6f1a5325d182e7a4257758876fe3e3a"}
Sep 08 14:04:47.206745 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:47.206693    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m" podStartSLOduration=2.206671786 podStartE2EDuration="2.206671786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:04:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:04:47.205264695 +0000 UTC m=+2173.423181038" watchObservedRunningTime="2026-09-08 14:04:47.206671786 +0000 UTC m=+2173.424588111"
Sep 08 14:04:58.193984 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:04:58.193950    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5b65f94d4c-8pc6m"
Sep 08 14:05:07.157484 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:07.157436    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:05:14.138101 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:14.138060    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:05:22.742492 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:22.742453    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:05:32.713782 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:32.713750    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:05:35.561866 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:35.561828    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:05:38.137763 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:38.137729    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:05:47.151049 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:47.151010    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:05:50.044715 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:05:50.044678    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:02.766359 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:02.766276    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:06:04.407119 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:04.407086    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:06:06.335925 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:06.335884    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:14.446725 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:14.446682    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:24.162049 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.162011    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"]
Sep 08 14:06:24.166877 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.166850    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.176540 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.176510    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"]
Sep 08 14:06:24.330745 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.330705    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-metrics-certs\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.330921 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.330775    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kgd47\" (UniqueName: \"kubernetes.io/projected/c20a7cc7-228b-4cdc-896e-c026ee584be8-kube-api-access-kgd47\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.330921 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.330866    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-maas-api-tls\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.330921 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.330907    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20a7cc7-228b-4cdc-896e-c026ee584be8-openshift-service-ca\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.431767 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.431679    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kgd47\" (UniqueName: \"kubernetes.io/projected/c20a7cc7-228b-4cdc-896e-c026ee584be8-kube-api-access-kgd47\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.431767 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.431745    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-maas-api-tls\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.431767 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.431770    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20a7cc7-228b-4cdc-896e-c026ee584be8-openshift-service-ca\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.432017 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.431818    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-metrics-certs\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.432564 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.432474    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c20a7cc7-228b-4cdc-896e-c026ee584be8-openshift-service-ca\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.434469 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.434438    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-metrics-certs\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.434601 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.434471    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c20a7cc7-228b-4cdc-896e-c026ee584be8-maas-api-tls\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.439579 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.439529    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:24.441228 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.441206    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kgd47\" (UniqueName: \"kubernetes.io/projected/c20a7cc7-228b-4cdc-896e-c026ee584be8-kube-api-access-kgd47\") pod \"maas-api-86dc7466b4-nw5dj\" (UID: \"c20a7cc7-228b-4cdc-896e-c026ee584be8\") " pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.479367 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.479333    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:24.754055 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.754027    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"]
Sep 08 14:06:24.756218 ip-10-0-140-23 kubenswrapper[2583]: W0908 14:06:24.756186    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc20a7cc7_228b_4cdc_896e_c026ee584be8.slice/crio-27898d54b745ced246a77e4089fbf88059efc0c69f7dedb07c03593c27a7cb0c WatchSource:0}: Error finding container 27898d54b745ced246a77e4089fbf88059efc0c69f7dedb07c03593c27a7cb0c: Status 404 returned error can't find the container with id 27898d54b745ced246a77e4089fbf88059efc0c69f7dedb07c03593c27a7cb0c
Sep 08 14:06:24.757903 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.757885    2583 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 08 14:06:24.791198 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:24.791162    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj" event={"ID":"c20a7cc7-228b-4cdc-896e-c026ee584be8","Type":"ContainerStarted","Data":"27898d54b745ced246a77e4089fbf88059efc0c69f7dedb07c03593c27a7cb0c"}
Sep 08 14:06:25.799191 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:25.799143    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj" event={"ID":"c20a7cc7-228b-4cdc-896e-c026ee584be8","Type":"ContainerStarted","Data":"f4baf92a8dbd37f6f86514c7faf2571e9833ebb6621262abbd198a94390c80a2"}
Sep 08 14:06:25.799609 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:25.799240    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:25.815946 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:25.815894    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj" podStartSLOduration=1.214113751 podStartE2EDuration="1.815874446s" totalImagesPullingTime="601.760695ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:06:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:06:25.815116577 +0000 UTC m=+2272.033032921" watchObservedRunningTime="2026-09-08 14:06:25.815874446 +0000 UTC m=+2272.033790773"
Sep 08 14:06:31.810982 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:31.810951    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-86dc7466b4-nw5dj"
Sep 08 14:06:33.857740 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:33.857696    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:39.590132 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:39.590089    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 14:06:39.590536 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:39.590320    2583 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" podUID="fc28ae82-9021-48d0-93db-0522b4f15253" containerName="maas-api" containerID="cri-o://c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76" gracePeriod=30
Sep 08 14:06:40.844511 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.844484    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 14:06:40.890954 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.890920    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls\") pod \"fc28ae82-9021-48d0-93db-0522b4f15253\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") "
Sep 08 14:06:40.891138 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.891037    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca\") pod \"fc28ae82-9021-48d0-93db-0522b4f15253\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") "
Sep 08 14:06:40.891138 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.891083    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g44mr\" (UniqueName: \"kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr\") pod \"fc28ae82-9021-48d0-93db-0522b4f15253\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") "
Sep 08 14:06:40.891268 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.891159    2583 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs\") pod \"fc28ae82-9021-48d0-93db-0522b4f15253\" (UID: \"fc28ae82-9021-48d0-93db-0522b4f15253\") "
Sep 08 14:06:40.891499 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.891476    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fc28ae82-9021-48d0-93db-0522b4f15253" (UID: "fc28ae82-9021-48d0-93db-0522b4f15253"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 08 14:06:40.893636 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.893601    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fc28ae82-9021-48d0-93db-0522b4f15253" (UID: "fc28ae82-9021-48d0-93db-0522b4f15253"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:06:40.893942 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.893911    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fc28ae82-9021-48d0-93db-0522b4f15253" (UID: "fc28ae82-9021-48d0-93db-0522b4f15253"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 08 14:06:40.893942 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.893931    2583 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr" (OuterVolumeSpecName: "kube-api-access-g44mr") pod "fc28ae82-9021-48d0-93db-0522b4f15253" (UID: "fc28ae82-9021-48d0-93db-0522b4f15253"). InnerVolumeSpecName "kube-api-access-g44mr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 08 14:06:40.896027 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.896004    2583 generic.go:356] "Generic (PLEG): container finished" podID="fc28ae82-9021-48d0-93db-0522b4f15253" containerID="c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76" exitCode=0
Sep 08 14:06:40.896148 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.896058    2583 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"
Sep 08 14:06:40.896148 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.896086    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" event={"ID":"fc28ae82-9021-48d0-93db-0522b4f15253","Type":"ContainerDied","Data":"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"}
Sep 08 14:06:40.896148 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.896134    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5" event={"ID":"fc28ae82-9021-48d0-93db-0522b4f15253","Type":"ContainerDied","Data":"a4bb210fc357b7acdd7bbf337d208a103d64fda8614a0e30866023505c7374f2"}
Sep 08 14:06:40.896306 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.896155    2583 scope.go:117] "RemoveContainer" containerID="c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"
Sep 08 14:06:40.915325 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.915304    2583 scope.go:117] "RemoveContainer" containerID="c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"
Sep 08 14:06:40.915772 ip-10-0-140-23 kubenswrapper[2583]: E0908 14:06:40.915749    2583 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76\": container with ID starting with c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76 not found: ID does not exist" containerID="c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"
Sep 08 14:06:40.915850 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.915781    2583 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76"} err="failed to get container status \"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76\": rpc error: code = NotFound desc = could not find container \"c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76\": container with ID starting with c79ef9f08d3f69bbb831cc0c49323bfa2fa8fa0c7525c6a032cd65ff2822bb76 not found: ID does not exist"
Sep 08 14:06:40.926260 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.926225    2583 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 14:06:40.929979 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.929949    2583 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-84d9df9944-2tvc5"]
Sep 08 14:06:40.992935 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.992856    2583 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-metrics-certs\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:06:40.992935 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.992883    2583 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc28ae82-9021-48d0-93db-0522b4f15253-maas-api-tls\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:06:40.992935 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.992892    2583 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc28ae82-9021-48d0-93db-0522b4f15253-openshift-service-ca\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:06:40.992935 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:40.992903    2583 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g44mr\" (UniqueName: \"kubernetes.io/projected/fc28ae82-9021-48d0-93db-0522b4f15253-kube-api-access-g44mr\") on node \"ip-10-0-140-23.ec2.internal\" DevicePath \"\""
Sep 08 14:06:41.048164 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:41.048131    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:42.231762 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:42.231724    2583 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc28ae82-9021-48d0-93db-0522b4f15253" path="/var/lib/kubelet/pods/fc28ae82-9021-48d0-93db-0522b4f15253/volumes"
Sep 08 14:06:49.357745 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:49.357710    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:06:55.037897 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:06:55.037861    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-85d9db4d5d-8nbgf"]
Sep 08 14:07:03.256373 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:03.256339    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x2vn9_3e84895b-aa10-40f5-8040-abd64742ec1e/manager/0.log"
Sep 08 14:07:03.380210 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:03.380158    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lwsx7_99c1fe38-ecb8-4c64-a0e1-f039b4650ae7/manager/0.log"
Sep 08 14:07:03.474669 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:03.474638    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:07:04.146818 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:04.146788    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-xc5lv_1f50e301-84d5-42f9-adf3-240aa20b5d3f/copy-manifests/0.log"
Sep 08 14:07:04.159658 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:04.159623    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-xc5lv_1f50e301-84d5-42f9-adf3-240aa20b5d3f/manager/0.log"
Sep 08 14:07:04.537185 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:04.537152    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-86dc7466b4-nw5dj_c20a7cc7-228b-4cdc-896e-c026ee584be8/maas-api/0.log"
Sep 08 14:07:06.036983 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.036951    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-zgwzk_82b6390a-5a75-423a-bcf5-3724ffd8c799/manager/0.log"
Sep 08 14:07:06.151525 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.151493    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-lg74b_9cd612c8-ddb4-4354-a95b-14621950f461/manager/0.log"
Sep 08 14:07:06.275764 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.275739    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-v44rd_63537a97-4519-468c-b170-c9c07f6a27eb/kuadrant-console-plugin/0.log"
Sep 08 14:07:06.406599 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.406500    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-8pc6m_71e22ddc-5cf5-464a-982a-afc300cf8206/manager/0.log"
Sep 08 14:07:06.540952 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.540925    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-8nbgf_2a240ca1-8435-4e0d-871b-d0cec30e1527/limitador/0.log"
Sep 08 14:07:06.662963 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:06.662874    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-5trd8_ae98832b-209c-4ed7-a1ee-f44cb0a99e0a/manager/0.log"
Sep 08 14:07:07.293256 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:07.293228    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55db547db7-6rxmr_4890981a-15b7-47ab-b279-b26bf5e02745/kube-auth-proxy/0.log"
Sep 08 14:07:08.372371 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:08.372342    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-5gkcp_d70e92b1-286b-423a-9967-afea26e104da/main/0.log"
Sep 08 14:07:08.857887 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:08.857857    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fks7zb_8ba31e5c-f07b-4d6a-ba04-765599932f6f/main/0.log"
Sep 08 14:07:14.322933 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.322891    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52jtx/must-gather-jk68m"]
Sep 08 14:07:14.323330 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.323322    2583 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc28ae82-9021-48d0-93db-0522b4f15253" containerName="maas-api"
Sep 08 14:07:14.323374 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.323333    2583 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc28ae82-9021-48d0-93db-0522b4f15253" containerName="maas-api"
Sep 08 14:07:14.323412 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.323400    2583 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc28ae82-9021-48d0-93db-0522b4f15253" containerName="maas-api"
Sep 08 14:07:14.326400 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.326382    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.334951 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.334926    2583 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-52jtx\"/\"default-dockercfg-dxk4h\""
Sep 08 14:07:14.334951 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.334937    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52jtx\"/\"openshift-service-ca.crt\""
Sep 08 14:07:14.335123 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.335022    2583 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-52jtx\"/\"kube-root-ca.crt\""
Sep 08 14:07:14.355726 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.355690    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/must-gather-jk68m"]
Sep 08 14:07:14.365273 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.365239    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:07:14.416913 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.416877    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/366fd91f-f1c4-4151-bdee-4d07baf0e47c-must-gather-output\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.417095 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.416937    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m6s95\" (UniqueName: \"kubernetes.io/projected/366fd91f-f1c4-4151-bdee-4d07baf0e47c-kube-api-access-m6s95\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.517784 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.517742    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/366fd91f-f1c4-4151-bdee-4d07baf0e47c-must-gather-output\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.517784 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.517789    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m6s95\" (UniqueName: \"kubernetes.io/projected/366fd91f-f1c4-4151-bdee-4d07baf0e47c-kube-api-access-m6s95\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.518104 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.518085    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/366fd91f-f1c4-4151-bdee-4d07baf0e47c-must-gather-output\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.525844 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.525820    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m6s95\" (UniqueName: \"kubernetes.io/projected/366fd91f-f1c4-4151-bdee-4d07baf0e47c-kube-api-access-m6s95\") pod \"must-gather-jk68m\" (UID: \"366fd91f-f1c4-4151-bdee-4d07baf0e47c\") " pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.635652 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.635539    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/must-gather-jk68m"
Sep 08 14:07:14.802012 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.801976    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:07:14.912760 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:14.912735    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/must-gather-jk68m"]
Sep 08 14:07:14.914117 ip-10-0-140-23 kubenswrapper[2583]: W0908 14:07:14.914090    2583 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod366fd91f_f1c4_4151_bdee_4d07baf0e47c.slice/crio-6f9d4a3124bd542bdf22aa28dd6d30148123e7ef3e8cb29d5058df9a29aab3b2 WatchSource:0}: Error finding container 6f9d4a3124bd542bdf22aa28dd6d30148123e7ef3e8cb29d5058df9a29aab3b2: Status 404 returned error can't find the container with id 6f9d4a3124bd542bdf22aa28dd6d30148123e7ef3e8cb29d5058df9a29aab3b2
Sep 08 14:07:15.109653 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:15.109616    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/must-gather-jk68m" event={"ID":"366fd91f-f1c4-4151-bdee-4d07baf0e47c","Type":"ContainerStarted","Data":"6f9d4a3124bd542bdf22aa28dd6d30148123e7ef3e8cb29d5058df9a29aab3b2"}
Sep 08 14:07:16.120418 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:16.120370    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/must-gather-jk68m" event={"ID":"366fd91f-f1c4-4151-bdee-4d07baf0e47c","Type":"ContainerStarted","Data":"866ed0fbe88862178211ac9d6727be4b8e2d27483d1246ed54604df6665b852c"}
Sep 08 14:07:17.133395 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.133358    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/must-gather-jk68m" event={"ID":"366fd91f-f1c4-4151-bdee-4d07baf0e47c","Type":"ContainerStarted","Data":"4b5636a1e40c45760ab0c1f459ac458c8fa780773140d99fd29a4ce86db70868"}
Sep 08 14:07:17.149483 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.149428    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52jtx/must-gather-jk68m" podStartSLOduration=2.077525047 podStartE2EDuration="3.149411236s" totalImagesPullingTime="1.071886189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-08 14:07:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:07:17.147988499 +0000 UTC m=+2323.365904825" watchObservedRunningTime="2026-09-08 14:07:17.149411236 +0000 UTC m=+2323.367327597"
Sep 08 14:07:17.297199 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.297167    2583 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log" limit=1024
Sep 08 14:07:17.596601 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.596568    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-ngbdv_7fad80d1-6663-4c00-a282-2815aad25ba7/global-pull-secret-syncer/0.log"
Sep 08 14:07:17.729496 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.729460    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ldrh2_13910b21-8b7c-46fa-b444-194c280bd51c/konnectivity-agent/0.log"
Sep 08 14:07:17.856829 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:17.856745    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-23.ec2.internal_b8ce096f9d3777e9f36fc81bb4086cc7/haproxy/0.log"
Sep 08 14:07:22.373474 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.373434    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-68d88948d8-zgwzk_82b6390a-5a75-423a-bcf5-3724ffd8c799/manager/0.log"
Sep 08 14:07:22.425070 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.422013    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-5985564776-lg74b_9cd612c8-ddb4-4354-a95b-14621950f461/manager/0.log"
Sep 08 14:07:22.454227 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.454178    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-598f86468b-v44rd_63537a97-4519-468c-b170-c9c07f6a27eb/kuadrant-console-plugin/0.log"
Sep 08 14:07:22.502015 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.501974    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5b65f94d4c-8pc6m_71e22ddc-5cf5-464a-982a-afc300cf8206/manager/0.log"
Sep 08 14:07:22.572419 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.572388    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-85d9db4d5d-8nbgf_2a240ca1-8435-4e0d-871b-d0cec30e1527/limitador/0.log"
Sep 08 14:07:22.675985 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:22.675899    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-6c549b6d85-5trd8_ae98832b-209c-4ed7-a1ee-f44cb0a99e0a/manager/0.log"
Sep 08 14:07:24.224854 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.224813    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/alertmanager/0.log"
Sep 08 14:07:24.261997 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.261966    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/config-reloader/0.log"
Sep 08 14:07:24.288137 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.288092    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/kube-rbac-proxy-web/0.log"
Sep 08 14:07:24.315133 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.315097    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/kube-rbac-proxy/0.log"
Sep 08 14:07:24.340216 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.340187    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/kube-rbac-proxy-metric/0.log"
Sep 08 14:07:24.369846 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.369812    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/prom-label-proxy/0.log"
Sep 08 14:07:24.402407 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.402372    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e7b8a25b-9ac9-4f8b-8cd0-c5c5b963d86f/init-config-reloader/0.log"
Sep 08 14:07:24.470312 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.470258    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-swj4m_fc15e85c-a68f-4adc-8940-2f4ee46da50c/kube-state-metrics/0.log"
Sep 08 14:07:24.491406 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.491285    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-swj4m_fc15e85c-a68f-4adc-8940-2f4ee46da50c/kube-rbac-proxy-main/0.log"
Sep 08 14:07:24.518305 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.518263    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-swj4m_fc15e85c-a68f-4adc-8940-2f4ee46da50c/kube-rbac-proxy-self/0.log"
Sep 08 14:07:24.582520 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.582493    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-tzcwg_b08705b1-29a2-47c9-af8f-b8ead9b25831/monitoring-plugin/0.log"
Sep 08 14:07:24.816170 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.816142    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7rp4_5da9d9e4-aaee-462b-be6d-5fdcf2703eef/node-exporter/0.log"
Sep 08 14:07:24.872963 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.872925    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7rp4_5da9d9e4-aaee-462b-be6d-5fdcf2703eef/kube-rbac-proxy/0.log"
Sep 08 14:07:24.899942 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:24.899909    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-z7rp4_5da9d9e4-aaee-462b-be6d-5fdcf2703eef/init-textfile/0.log"
Sep 08 14:07:25.321674 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.321576    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-758c7d646f-2rl47_acb517ae-2729-4d6a-b4ca-e0a63ffc77fa/telemeter-client/0.log"
Sep 08 14:07:25.348602 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.348573    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-758c7d646f-2rl47_acb517ae-2729-4d6a-b4ca-e0a63ffc77fa/reload/0.log"
Sep 08 14:07:25.372356 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.372327    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-758c7d646f-2rl47_acb517ae-2729-4d6a-b4ca-e0a63ffc77fa/kube-rbac-proxy/0.log"
Sep 08 14:07:25.417750 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.417687    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/thanos-query/0.log"
Sep 08 14:07:25.447732 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.447701    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/kube-rbac-proxy-web/0.log"
Sep 08 14:07:25.473392 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.473368    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/kube-rbac-proxy/0.log"
Sep 08 14:07:25.502442 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.502410    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/prom-label-proxy/0.log"
Sep 08 14:07:25.528917 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.528885    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/kube-rbac-proxy-rules/0.log"
Sep 08 14:07:25.553898 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:25.553871    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6c5b7746c4-rx96s_4b1305f4-62ba-43e0-86d4-a940c40610e0/kube-rbac-proxy-metrics/0.log"
Sep 08 14:07:27.029624 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:27.029589    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-79977f9844-g5c67_308c88fa-8f8c-4646-8c3a-8bc55633bbc1/networking-console-plugin/0.log"
Sep 08 14:07:28.170128 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:28.170082    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-f85d669c7-f7zwh_39cddde0-5706-4ced-8b0c-fbc7498123ba/console/0.log"
Sep 08 14:07:28.202965 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:28.202924    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-dfvfg_973a6fba-efe8-4903-8e80-db8ce1a831d1/download-server/0.log"
Sep 08 14:07:29.172632 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.172576    2583 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"]
Sep 08 14:07:29.181474 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.181438    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.185945 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.185853    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"]
Sep 08 14:07:29.282076 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.282033    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-proc\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.282284 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.282088    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-podres\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.282284 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.282140    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-sys\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.282284 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.282200    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gnsc8\" (UniqueName: \"kubernetes.io/projected/acc45649-1032-4316-8264-c5d6054ac4f1-kube-api-access-gnsc8\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.282456 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.282287    2583 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-lib-modules\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.383740 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.383698    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-podres\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.383936 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.383802    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-sys\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.383936 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.383867    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gnsc8\" (UniqueName: \"kubernetes.io/projected/acc45649-1032-4316-8264-c5d6054ac4f1-kube-api-access-gnsc8\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.383936 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.383891    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-podres\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.384099 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.383979    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-lib-modules\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.384099 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.384082    2583 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-proc\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.384226 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.384204    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-proc\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.384338 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.384314    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-lib-modules\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.384338 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.384315    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/acc45649-1032-4316-8264-c5d6054ac4f1-sys\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.393794 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.393764    2583 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gnsc8\" (UniqueName: \"kubernetes.io/projected/acc45649-1032-4316-8264-c5d6054ac4f1-kube-api-access-gnsc8\") pod \"perf-node-gather-daemonset-7269b\" (UID: \"acc45649-1032-4316-8264-c5d6054ac4f1\") " pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.501169 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.501089    2583 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:29.811236 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:29.811189    2583 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"]
Sep 08 14:07:30.141827 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.141748    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2j96_0e7a9712-c9fa-4a81-8bee-304a282f05b6/dns/0.log"
Sep 08 14:07:30.163533 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.163497    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g2j96_0e7a9712-c9fa-4a81-8bee-304a282f05b6/kube-rbac-proxy/0.log"
Sep 08 14:07:30.284348 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.284324    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ljkxh_7e942d48-6e53-498f-8d2b-b9f88ca36d83/dns-node-resolver/0.log"
Sep 08 14:07:30.289862 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.289828    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b" event={"ID":"acc45649-1032-4316-8264-c5d6054ac4f1","Type":"ContainerStarted","Data":"0ad4c5d57a6bfd2df1d77ff911340fb480f55926486f74eabb237e29948638da"}
Sep 08 14:07:30.290030 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.289891    2583 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b" event={"ID":"acc45649-1032-4316-8264-c5d6054ac4f1","Type":"ContainerStarted","Data":"70d5450ed79bc4c5f6875f1f6f78e7d1991ee6a01d0e97552fa7ca529e614c0b"}
Sep 08 14:07:30.290096 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.290059    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:30.307052 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.306999    2583 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b" podStartSLOduration=1.3069818180000001 podStartE2EDuration="1.306981818s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-08 14:07:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-08 14:07:30.304356533 +0000 UTC m=+2336.522272877" watchObservedRunningTime="2026-09-08 14:07:30.306981818 +0000 UTC m=+2336.524898137"
Sep 08 14:07:30.905714 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.905678    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-74b8f69c69-f4p98_dce2af13-dcad-4d45-bef6-78745a498bb3/registry/0.log"
Sep 08 14:07:30.981275 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:30.981244    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lt4dq_60a8bc96-a399-4cd8-9ebf-a0499e48b232/node-ca/0.log"
Sep 08 14:07:32.232844 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:32.232797    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55db547db7-6rxmr_4890981a-15b7-47ab-b279-b26bf5e02745/kube-auth-proxy/0.log"
Sep 08 14:07:33.270972 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.270943    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-5lhct_1149f2f0-504e-4f23-9a7e-893aa93405d7/serve-healthcheck-canary/0.log"
Sep 08 14:07:33.799082 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.799047    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-zwcmh_bfb83340-d2b2-4688-89c8-5e55a96c0d01/insights-operator/0.log"
Sep 08 14:07:33.802899 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.802864    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-76bcd6ccf5-zwcmh_bfb83340-d2b2-4688-89c8-5e55a96c0d01/insights-operator/1.log"
Sep 08 14:07:33.903026 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.902978    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsjt9_d4635088-a268-4e79-8ae3-ebaf77e812e1/kube-rbac-proxy/0.log"
Sep 08 14:07:33.925300 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.925267    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsjt9_d4635088-a268-4e79-8ae3-ebaf77e812e1/exporter/0.log"
Sep 08 14:07:33.947226 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:33.947198    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-wsjt9_d4635088-a268-4e79-8ae3-ebaf77e812e1/extractor/0.log"
Sep 08 14:07:36.192815 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:36.192783    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-x2vn9_3e84895b-aa10-40f5-8040-abd64742ec1e/manager/0.log"
Sep 08 14:07:36.312286 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:36.311999    2583 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-52jtx/perf-node-gather-daemonset-7269b"
Sep 08 14:07:36.436879 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:36.436837    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-lwsx7_99c1fe38-ecb8-4c64-a0e1-f039b4650ae7/manager/0.log"
Sep 08 14:07:36.786795 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:36.786756    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-xc5lv_1f50e301-84d5-42f9-adf3-240aa20b5d3f/manager/0.log"
Sep 08 14:07:36.805736 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:36.805710    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-54685d959b-xc5lv_1f50e301-84d5-42f9-adf3-240aa20b5d3f/copy-manifests/0.log"
Sep 08 14:07:38.108987 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:38.108911    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-cpsp9_51ab330e-d91c-44f2-928a-8fe4f2adf108/openshift-lws-operator/0.log"
Sep 08 14:07:39.182933 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:39.182896    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55db547db7-6rxmr_4890981a-15b7-47ab-b279-b26bf5e02745/kube-auth-proxy/0.log"
Sep 08 14:07:41.151913 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:41.151842    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-55db547db7-6rxmr_4890981a-15b7-47ab-b279-b26bf5e02745/kube-auth-proxy/0.log"
Sep 08 14:07:45.212625 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.212595    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-94lfj_22924846-24ee-409a-a0fe-2acdc375fa2b/kube-multus/0.log"
Sep 08 14:07:45.236081 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.236055    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/kube-multus-additional-cni-plugins/0.log"
Sep 08 14:07:45.258386 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.258350    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/egress-router-binary-copy/0.log"
Sep 08 14:07:45.281418 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.281381    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/cni-plugins/0.log"
Sep 08 14:07:45.306869 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.306829    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/bond-cni-plugin/0.log"
Sep 08 14:07:45.330522 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.330494    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/routeoverride-cni/0.log"
Sep 08 14:07:45.353581 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.353539    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/whereabouts-cni-bincopy/0.log"
Sep 08 14:07:45.375600 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.375546    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-chtg8_91344415-9650-42ef-a460-f8a3029a9100/whereabouts-cni/0.log"
Sep 08 14:07:45.863507 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.863477    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-msdpw_b10de668-df4d-40b5-9e19-255843710763/network-metrics-daemon/0.log"
Sep 08 14:07:45.888211 ip-10-0-140-23 kubenswrapper[2583]: I0908 14:07:45.888187    2583 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-msdpw_b10de668-df4d-40b5-9e19-255843710763/kube-rbac-proxy/0.log"